To execute the commands properly you must have the RFTagger downloaded and unzipped. Always execute the commands on the directory where you have the texts you want to edit. Make sure you have the corpus directory inside the RFTagger directory.

XML to Text:

$ for f in *.xml; do grep -Po ‘<.*?>/K.*?(?=<.*?>)’<$f> $f.tmp; done

Rename .tmp to .txt:

$ rename .xml.tmp .txt *.xml.tmp

Tokenize (not needed with linux):

$ for f in *.txt; do ~/RFTagger/cmd/tokenize.perl $f > $f.tmp; done

Annotate with RFTagger:

$ for f in .txt; do ~/RFTagger/src/rft-annotate.exe ~/RFTagger/lib/german-latin1.par $f > $f.tmp; done

POS only (need tagged texts):

$ for f in *.txt; do awk ‘{print $2}’ $f > $f.tmp; done

Replace periods with i:

$ sed -i.tmp ‘s/\./i/g’ *.txt

The replacement is needed if you want to tokenize the POS only corpus, because the tokenizer would break up the Tags at the periods.

Extract lines with ADJ (adjectives):

$ find . -name “*.txt” -type f -exec sh -c ‘grep “ADJ” < “$0”>”0.txt”’ {}\;

Extract the words after extracting ADJ (and renaming to .txt):

$ for f in *.txt; do awk ‘{print $1}’ $f > $f.tmp; done

Same for the other POS-Tags. Note that this command searches the string ‘ADJ’ and would search any other string in the text. If you look for ART it will also find APPART.

XML run 1: MFW 50, Culled 90%, Pronouns deleted, Started at 0
XML run 2: MFW 700, Culled 0%, Pronouns deleted, Started at 100
POS only run 1: MFW 30, Culled 80%, Pronouns included, Started at 0
POS only run 2: MFW 269, Culled 0%, Pronouns included, Started at 0
ADJ only: MFW 700, Culled 0%, Pronouns included, Started at 0
N only: MFW 100, Culled 80%, Pronouns included, Started at 15
VINF only: MFW 1500, Culled 0%, Pronouns included, Started at 0

To execute the commands properly you must have the RFTagger downloaded and unzipped. Always execute the commands on the directory where you have the texts you want to edit. Make sure you have the corpus directory inside the RFTagger directory.

XML to Text:

$ for f in *.xml; do grep -Po ‘<.*?>/K.*?(?=<.*?>)’<$f> $f.tmp; done

Rename .tmp to .txt:

$ rename .xml.tmp .txt *.xml.tmp

Tokenize (not needed with linux):

$ for f in *.txt; do ~/RFTagger/cmd/tokenize.perl $f > $f.tmp; done

Annotate with RFTagger:

$ for f in .txt; do ~/RFTagger/src/rft-annotate.exe ~/RFTagger/lib/german-latin1.par $f > $f.tmp; done

POS only (need tagged texts):

$ for f in *.txt; do awk ‘{print $2}’ $f > $f.tmp; done

Replace periods with i:

$ sed -i.tmp ‘s/\./i/g’ *.txt

The replacement is needed if you want to tokenize the POS only corpus, because the tokenizer would break up the Tags at the periods.

Extract lines with ADJ (adjectives):

$ find . -name “*.txt” -type f -exec sh -c ‘grep “ADJ” < “$0”>”0.txt”’ {}\;

Extract the words after extracting ADJ (and renaming to .txt):

$ for f in *.txt; do awk ‘{print $1}’ $f > $f.tmp; done

Same for the other POS-Tags. Note that this command searches the string ‘ADJ’ and would search any other string in the text. If you look for ART it will also find APPART.

XML run 1: MFW 50, Culled 90%, Pronouns deleted, Started at 0
XML run 2: MFW 700, Culled 0%, Pronouns deleted, Started at 100
POS only run 1: MFW 30, Culled 80%, Pronouns included, Started at 0
POS only run 2: MFW 269, Culled 0%, Pronouns included, Started at 0
ADJ only: MFW 700, Culled 0%, Pronouns included, Started at 0
N only: MFW 100, Culled 80%, Pronouns included, Started at 15
VINF only: MFW 1500, Culled 0%, Pronouns included, Started at 0