Skip to content

Commit d9419e8

Browse files
committed
added data generator note
1 parent 721e627 commit d9419e8

File tree

1 file changed

+5
-3
lines changed

1 file changed

+5
-3
lines changed

CNN_kanji/jupyter/DaKanjiRecognizer.ipynb

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,6 @@
3737
"from etldr.etl_data_reader import ETLDataReader\n",
3838
"from etldr.etl_character_groups import ETLCharacterGroups\n",
3939
"from etldr.etl_data_names import ETLDataNames\n",
40-
"import DataGenerator\n",
4140
"\n",
4241
"#data handling\n",
4342
"import PIL\n",
@@ -496,10 +495,11 @@
496495
"source": [
497496
"## Defining a thread-safe Datagenerator\n",
498497
"\n",
499-
"**caution: on windows multiprocesing in jupyter is difficult. For actual training use `training.py`** on windwos.\n",
498+
"**caution: on windows multiprocesing in jupyter is difficult. For actual training use `training.py`** on windows.\n",
500499
"\n",
501500
"Because the data set is too large to fit into memory we will first generate a data generator to create batches on the fly. <br> \n",
502-
"Instantiate a generator for training and testing."
501+
"Instantiate a generator for training and testing. <br>\n",
502+
"This generator uses multi processing. Multi processing does not work in jupyter notebooks (on windows) when defined in the notebook. Therefore the datagenerator is implemented in `DataGenerator.py`."
503503
]
504504
},
505505
{
@@ -508,6 +508,8 @@
508508
"metadata": {},
509509
"outputs": [],
510510
"source": [
511+
"import DataGenerator\n",
512+
"\n",
511513
"train_generator = DataGenerator.DataGenerator(len(x_np), 2048, \n",
512514
" 80, \"training\", \n",
513515
" x_shared, y_shared, \n",

0 commit comments

Comments
 (0)