More combined models?

[amir-zeldes]

I saw the great idea for combined models here:

https://stanfordnlp.github.io/stanza/combined_models.html

Is there a process to request more of these? Specifically I was thinking of Hebrew right now.

[AngledLuffa]

For English, we did the best we can to unify the xpos tags features. Same with Italian. That was a bit more difficult, since no one on the project speaks Italian, but the treebank maintainers were quite helpful with that.

If the Hebrew standards are the same across different treebanks, we can mix them together and see what happens. I assume you're talking about IAHLT and HTB?

[AngledLuffa]

There's also Hebrew NER and constituency datasets which can be added, if you want to see more Hebrew stuff in general in Stanza. Sentiment dataset here:

https://github.com/OnlpLab/Hebrew-Sentiment-Data

Overall, a bunch of stuff we could add if we took some effort to improve our Hebrew pipeline.

[AngledLuffa]

I took a look at the two treebanks.

First thing to note is that they seem to follow similar MWT splitting guidelines, although I don't read Hebrew so I don't know how similar they are. Would you double check that?

I can see the xpos tags are the same as the upos in both treebanks, so no questions there. It appears roughly the same fraction of words are featurized in both sets. However, there are some differences between the two sets:

Feature differences:

In IAHLT, not HTB:

Aspect=Prog
Foreign=Yes
HebBinyan=NITPAEL
NumType=Card
NumType=Ord
Poss=Yes
Tense=Pres

in HTB, not IAHLT:

Case=Tem
HebExistential=Yes
Number=Dual,Plur
Person=1,2,3
Typo=Yes

If this is something you can help unify, we'll be happy to make combined models.

[amir-zeldes]

Actually the segmentation guidelines of the old HTB don't match IAHLT, and as you noted there are feature differences. In fact, the HTB in the UD repo hasn't been valid since 2018, so it's in legacy status and doesn't match what's in IAHLT, which is newer. However we do have a revised version of HTB which is valid (at least as of UD2.9) and matches the standards in IAHLTwiki pretty closely. You can find it here:

https://github.com/IAHLT/UD_Hebrew

That should allow for good joint results, combined with https://github.com/universalDependencies/UD_Hebrew-IAHLTWiki

[AngledLuffa]

That's interesting. Is there a possibility of upgrading the UD version of HTB to the revised version? I haven't been following developments for those treebanks at all.

[amir-zeldes]

I think the earliest that could happen would be in November, based on the guidelines here:

http://quest.ms.mff.cuni.cz/udvalidator/cgi-bin/unidep/validation-report.pl

The preamble states that after 4 years in legacy status, an invalid legacy TB will be excluded from the release, so perhaps this would be a chance to propose switching to that fork.

[AngledLuffa]

I should be able to make this happen tomorrow.

[amir-zeldes]

Fantastic, just let me know once it's available - it's too late for something I needed for an author response, but could still go into a camera ready version... :)

[AngledLuffa]

Would have made it a bit more of a priority if I knew there was a clock. It takes a few hours to go from 0 to models, and I was using my time (and my GPU time) to make some progress on the sentiment classifier and parser.

Anyway, I just need to finish up the depparse and it will be ready. (This has to happen after POS because we use the tagger to use predicted tags.) Are there any other models which would be useful, such as NER, constituency, or sentiment?

[AngledLuffa]

Ok, if you use stanza dev branch, it should be available now.

I'm gonna make a new release soon with it as default - just need to retrain a few models first

[amir-zeldes]

Oh wow, thank you! And no worries about the clock, this was super fast! We can include numbers from this model in our upcoming paper now.

@ivrit @yifatbm @nlhowell @AvnerAlgom - note this means that there will be a joint wiki+HTB Hebrew Stanza model using the new tokenization out of the box

[AngledLuffa]

This is part of stanza as of 1.4.1

Do you plan on taking over maintenance of the original HTB (or at least requesting it) when UD 2.11 comes out? Currently this is kind of ad-hoc, needing a couple different items downloaded first before building the models.