@@ -64,13 +64,13 @@ syntaxnet
64
64
65
65
### how to test
66
66
``` shell
67
- ( after installing syntaxnet)
67
+ # after installing syntaxnet
68
68
$ pwd
69
69
/path/to/models/syntaxnet
70
70
$ git clone https://github.com/dsindex/syntaxnet.git work
71
71
$ cd work
72
72
$ echo " hello syntaxnet" | ./demo.sh
73
- ( training parser only with parsed corpus)
73
+ # training parser only with parsed corpus
74
74
$ ./parser_trainer_test.sh
75
75
```
76
76
@@ -79,16 +79,16 @@ $ ./parser_trainer_test.sh
79
79
$ cd work
80
80
$ mkdir corpus
81
81
$ cd corpus
82
- ( downloading ud-treebanks-v2.0.tgz)
82
+ # downloading ud-treebanks-v2.0.tgz
83
83
$ tar -zxvf ud-treebanks-v2.0.tgz
84
84
$ ls universal-dependencies-2.0
85
85
$ UD_Ancient_Greek UD_Basque UD_Czech ....
86
86
```
87
87
88
88
### training tagger and parser with another corpus
89
89
``` shell
90
- ( for example, training UD_English)
91
- ( detail instructions can be found in https://github.com/tensorflow/models/tree/master/syntaxnet)
90
+ # for example, training UD_English
91
+ # detail instructions can be found in https://github.com/tensorflow/models/tree/master/syntaxnet
92
92
$ ./train.sh -v -v
93
93
...
94
94
# preprocessing with tagger
@@ -116,8 +116,7 @@ INFO:tensorflow:Seconds elapsed in evaluation: 34.97, eval metric: 83.49%
116
116
117
117
### training parser only
118
118
``` shell
119
- (in case you have other pos-tagger
120
- and want to build parser only from the parsed corpus)
119
+ # in case you have other pos-tagger and want to build parser only from the parsed corpus
121
120
$ ./train_p.sh -v -v
122
121
...
123
122
# pretrain parser
@@ -150,7 +149,7 @@ tagger NN ROOT
150
149
+-- and CC cc
151
150
+-- parser NN conj
152
151
153
- * original model
152
+ # original model
154
153
$ echo " this is my own tagger and parser" | ./demo.sh
155
154
Input: this is my own tagger and parser
156
155
Parse:
@@ -173,7 +172,7 @@ brought VBD ROOT
173
172
| +-- to IN case
174
173
+-- . . punct
175
174
176
- * original model
175
+ # original model
177
176
$ echo " Bob brought the pizza to Alice ." | ./demo.sh
178
177
Input: Bob brought the pizza to Alice .
179
178
Parse:
@@ -272,7 +271,7 @@ Parse:
272
271
` ` `
273
272
# ## apply korean POS tagger(Komoran via konlpy)
274
273
` ` ` shell
275
- * install konlpy ( http://konlpy.org/ko/v0.4.3/ )
274
+ # after installing konlpy ( http://konlpy.org/ko/v0.4.3/ )
276
275
$ python sejong/tagger.py
277
276
나는 학교에 간다.
278
277
1 나 나 NP NP _ 0 _ _ _
@@ -316,6 +315,7 @@ Parsing :
316
315
` ` ` shell
317
316
# download models from http://download.tensorflow.org/models/parsey_universal/<language>.zip
318
317
318
+ # for `English`
319
319
$ echo " Bob brought the pizza to Alice." | ./parse.sh
320
320
321
321
# tokenizing
@@ -390,23 +390,16 @@ the hyperparameters for POS tagger :
390
390
$ cd ../
391
391
$ pwd
392
392
/path/to/models/syntaxnet
393
- $ bazel --output_user_root=bazel_root build --nocheck_visibility - c opt -s //examples/dragnn:tutorial_1 --genrule_strategy=standalone --spawn_strategy=standalone --verbose_failures
393
+ $ bazel build -c opt //examples/dragnn:tutorial_1
394
394
` ` `
395
395
- training tagger and parser with CoNLL corpus
396
396
` ` `
397
- * prepare corpus
398
- $ mkdir -p work/dragnn_examples/data
399
- $ cp -rf work/UD_English/en-ud-train.conllu work/dragnn_examples/data
400
- $ cp -rf work/UD_English/en-ud-dev.conllu work/dragnn_examples/data
401
- $ python work/convert.py < work/dragnn_examples/data/en-ud-train.conllu > work/dragnn_examples/data/en-ud-train.conllu.conv
402
- $ python work/convert.py < work/dragnn_examples/data/en-ud-dev.conllu > work/dragnn_examples/data/en-ud-dev.conllu.conv
403
-
404
- $ cp -rf work/dragnn_examples/BUILD examples/dragnn/
405
- $ cp -rf work/dragnn_examples/test_dragnn.py examples/dragnn/
406
- $ bazel --output_user_root=bazel_root build --nocheck_visibility -c opt -s //examples/dragnn:test_dragnn --genrule_strategy=standalone --spawn_strategy=standalone --verbose_failures
407
- $ ./bazel-bin/examples/dragnn/test_dragnn --data_dir=work/dragnn_examples/data --train_file=work/dragnn_examples/data/en-ud-train.conllu.conv --dev_file=work/dragnn_examples/data/en-ud-dev.conllu.conv
408
- * trained model
409
- $ ls -al ls -al work/dragnn_examples/data
397
+ $ pwd
398
+ /path/to/models/syntaxnet/work
399
+ $ ./dragnn.sh -v -v
400
+ ` ` `
401
+ - test CoNLL2017 baseline
402
+ ` ` `
410
403
` ` `
411
404
- docker version
412
405
- [dragnn/syntaxnet docker](https://hub.docker.com/r/tensorflow/syntaxnet/)
0 commit comments