Modele

Semantyka dystrybucyjna znalazła w ostatnich latach szerokie zastosowanie w rozwiązywaniu szeregu zadań związanych z przetwarzaniem języka naturalnego. U jej podstaw leży hipoteza, że słowa występujące w podobnych kontekstach w dużych zbiorach danych tekstowych mają podobne znaczenie. Znaczenia słów reprezentowane są przez wektory liczbowe.

Poniższa lista zawiera modele dystrybucyjne (zbiory wektorów reprezentujących znaczenia słów) dla języka polskiego wygenerowane przy pomocy pakietu gensim dla dwóch dużych korpusów tekstowych: pełnej wersji Narodowego Korpusu Języka Polskiego (NKJP) oraz polskiej edycji Wikipedii z końca 2016 roku.

Źródło:
Typ modelu:
Rozmiar wektora:
# Nazwa pliku do pobrania MB
1 nkjp+wiki-forms-all-100-cbow-hs.txt.gz 802.3
2 nkjp+wiki-forms-all-100-cbow-ns.txt.gz 758.6
3 nkjp+wiki-forms-all-100-skipg-hs.txt.gz 780.0
4 nkjp+wiki-forms-all-100-skipg-ns.txt.gz 771.0
5 nkjp+wiki-forms-all-300-cbow-hs-50.txt.gz 634.3
6 nkjp+wiki-forms-all-300-cbow-hs.txt.gz 2312.1
7 nkjp+wiki-forms-all-300-cbow-ns-50.txt.gz 620.8
8 nkjp+wiki-forms-all-300-cbow-ns.txt.gz 2172.2
9 nkjp+wiki-forms-all-300-skipg-hs-50.txt.gz 590.2
10 nkjp+wiki-forms-all-300-skipg-hs.txt.gz 2242.1
11 nkjp+wiki-forms-all-300-skipg-ns-50.txt.gz 591.4
12 nkjp+wiki-forms-all-300-skipg-ns.txt.gz 2197.3
13 nkjp+wiki-forms-restricted-100-cbow-hs.txt.gz 748.3
14 nkjp+wiki-forms-restricted-100-cbow-ns.txt.gz 707.2
15 nkjp+wiki-forms-restricted-100-skipg-hs.txt.gz 727.3
16 nkjp+wiki-forms-restricted-100-skipg-ns.txt.gz 720.3
17 nkjp+wiki-forms-restricted-300-cbow-hs.txt.gz 2158.0
18 nkjp+wiki-forms-restricted-300-cbow-ns.txt.gz 2026.6
19 nkjp+wiki-forms-restricted-300-skipg-hs.txt.gz 2094.1
20 nkjp+wiki-forms-restricted-300-skipg-ns.txt.gz 2052.9
21 nkjp+wiki-lemmas-all-100-cbow-hs.txt.gz 580.4
22 nkjp+wiki-lemmas-all-100-cbow-ns.txt.gz 548.0
23 nkjp+wiki-lemmas-all-100-skipg-hs.txt.gz 569.1
24 nkjp+wiki-lemmas-all-100-skipg-ns.txt.gz 560.2
25 nkjp+wiki-lemmas-all-300-cbow-hs.txt.gz 1673.2
26 nkjp+wiki-lemmas-all-300-cbow-ns-50.txt.gz 325.7
27 nkjp+wiki-lemmas-all-300-cbow-ns.txt.gz 1568.3
28 nkjp+wiki-lemmas-all-300-skipg-hs.txt.gz 1632.8
29 nkjp+wiki-lemmas-all-300-skipg-ns.txt.gz 1594.0
30 nkjp+wiki-lemmas-pos-all-300-cbow-hs.txt.gz 1822.7
31 nkjp+wiki-lemmas-pos-all-300-skipg-ns.txt.gz 1737.3
32 nkjp+wiki-lemmas-restricted-100-cbow-ns.txt.gz 497.7
33 nkjp+wiki-lemmas-restricted-100-skipg-hs.txt.gz 517.0
34 nkjp+wiki-lemmas-restricted-100-skipg-ns.txt.gz 509.5
35 nkjp+wiki-lemmas-restricted-300-cbow-hs.txt.gz 1521.2
36 nkjp+wiki-lemmas-restricted-300-cbow-ns.txt.gz 1424.9
37 nkjp+wiki-lemmas-restricted-300-skipg-hs.txt.gz 1484.7
38 nkjp+wiki-lemmas-restricted-300-skipg-ns.txt.gz 1449.7
39 nkjp-forms-all-100-cbow-hs.txt.gz 693.2
40 nkjp-forms-all-100-cbow-ns.txt.gz 656.6
41 nkjp-forms-all-100-skipg-hs.txt.gz 672.7
42 nkjp-forms-all-100-skipg-ns.txt.gz 665.6
43 nkjp-forms-all-300-cbow-hs.txt.gz 1997.9
44 nkjp-forms-all-300-cbow-ns.txt.gz 1880.7
45 nkjp-forms-all-300-skipg-hs.txt.gz 1934.5
46 nkjp-forms-all-300-skipg-ns.txt.gz 1897.2
47 nkjp-forms-restricted-100-cbow-hs.txt.gz 647.3
48 nkjp-forms-restricted-100-cbow-ns.txt.gz 613.4
49 nkjp-forms-restricted-100-skipg-hs.txt.gz 628.7
50 nkjp-forms-restricted-100-skipg-ns.txt.gz 622.7
51 nkjp-forms-restricted-300-cbow-hs.txt.gz 1867.7
52 nkjp-forms-restricted-300-cbow-ns.txt.gz 1757.1
53 nkjp-forms-restricted-300-skipg-hs.txt.gz 1809.4
54 nkjp-forms-restricted-300-skipg-ns.txt.gz 1775.0
55 nkjp-lemmas-all-100-cbow-hs.txt.gz 480.9
56 nkjp-lemmas-all-100-cbow-ns.txt.gz 454.6
57 nkjp-lemmas-all-100-skipg-hs.txt.gz 471.0
58 nkjp-lemmas-all-100-skipg-ns.txt.gz 463.7
59 nkjp-lemmas-all-300-cbow-hs.txt.gz 1386.0
60 nkjp-lemmas-all-300-cbow-ns.txt.gz 1301.6
61 nkjp-lemmas-all-300-skipg-hs.txt.gz 1351.4
62 nkjp-lemmas-all-300-skipg-ns.txt.gz 1319.3
63 nkjp-lemmas-restricted-100-cbow-hs.txt.gz 436.1
64 nkjp-lemmas-restricted-100-cbow-ns.txt.gz 412.2
65 nkjp-lemmas-restricted-100-skipg-hs.txt.gz 426.9
66 nkjp-lemmas-restricted-100-skipg-ns.txt.gz 420.6
67 nkjp-lemmas-restricted-300-cbow-hs.txt.gz 1257.2
68 nkjp-lemmas-restricted-300-cbow-ns.txt.gz 1180.3
69 nkjp-lemmas-restricted-300-skipg-hs.txt.gz 1225.7
70 nkjp-lemmas-restricted-300-skipg-ns.txt.gz 1197.4
71 wiki-forms-all-100-cbow-hs.txt.gz 272.8
72 wiki-forms-all-100-cbow-ns-30-it100.txt.gz 91.8
73 wiki-forms-all-100-cbow-ns.txt.gz 258.8
74 wiki-forms-all-100-skipg-hs.txt.gz 266.6
75 wiki-forms-all-100-skipg-ns-30-it100.txt.gz 84.7
76 wiki-forms-all-100-skipg-ns.txt.gz 263.7
77 wiki-forms-all-300-cbow-hs-30.txt.gz 254.8
78 wiki-forms-all-300-cbow-hs.txt.gz 786.3
79 wiki-forms-all-300-cbow-ns-30.txt.gz 246.2
80 wiki-forms-all-300-cbow-ns.txt.gz 737.7
81 wiki-forms-all-300-skipg-hs.txt.gz 765.7
82 wiki-forms-all-300-skipg-ns.txt.gz 750.1
83 wiki-forms-restricted-100-cbow-hs.txt.gz 262.4
84 wiki-forms-restricted-100-cbow-ns.txt.gz 248.8
85 wiki-forms-restricted-100-skipg-hs.txt.gz 256.8
86 wiki-forms-restricted-100-skipg-ns.txt.gz 253.8
87 wiki-forms-restricted-300-cbow-hs.txt.gz 756.9
88 wiki-forms-restricted-300-cbow-ns.txt.gz 708.6
89 wiki-forms-restricted-300-skipg-hs.txt.gz 737.9
90 wiki-forms-restricted-300-skipg-ns.txt.gz 722.6
91 wiki-lemmas-all-100-cbow-hs.txt.gz 177.2
92 wiki-lemmas-all-100-cbow-ns.txt.gz 167.7
93 wiki-lemmas-all-100-skipg-hs.txt.gz 173.8
94 wiki-lemmas-all-100-skipg-ns.txt.gz 171.3
95 wiki-lemmas-all-300-cbow-hs.txt.gz 510.6
96 wiki-lemmas-all-300-cbow-ns.txt.gz 479.3
97 wiki-lemmas-all-300-skipg-hs.txt.gz 498.7
98 wiki-lemmas-all-300-skipg-ns.txt.gz 487.3
99 wiki-lemmas-restricted-100-cbow-hs.txt.gz 167.3
100 wiki-lemmas-restricted-100-cbow-ns.txt.gz 158.1
101 wiki-lemmas-restricted-100-skipg-hs.txt.gz 164.2
102 wiki-lemmas-restricted-100-skipg-ns.txt.gz 161.9
103 wiki-lemmas-restricted-300-cbow-hs.txt.gz 481.9
104 wiki-lemmas-restricted-300-cbow-ns.txt.gz 451.7
105 wiki-lemmas-restricted-300-skipg-hs.txt.gz 471.3
106 wiki-lemmas-restricted-300-skipg-ns.txt.gz 460.5
Nazwa pliku: corpus-type-stype-dim-arch-alg.txt.gz
corpus nazwa korpusu - nkjp, wiki lub nkjp+wiki
type typ modelu - model oparty na formach (forms)
lub lematach (lemmas)
stype podtyp modelu - wszystkie części mowy (all)
lub tylko wybrane części mowy (restricted)
dim rozmiar wektora - 100 lub 300
arch architektura sieci neuronowej - CBOW (cbow)
lub Skip-Gram (skipg)
alg algorytm uczący - Hierarchical Softmax (hs)
lub Negative Sampling (ns)
Niektóre modele ograniczone zostały tylko do tych słów,
które wystąpiły co najmniej 30 lub 50 razy w korpusie.
Jest to zaznaczone po nazwie algorytmu uczącego alg.
it100 w nazwie pliku oznacza, że dany model został
wytrenowany w stu iteracjach.

Wybrane modele w postaci binarnej:
nkjp+wiki-forms-all-300-skipg-hs-50.zip
nkjp+wiki-lemmas-all-300-cbow-ns-50.zip
nkjp+wiki-forms-all-100-skipg-ns.zip