This post aims to setup environment for  ‘TensorFlow for Poets’ case but with slight adaptations for Windows (7) environment. You can find the linux version here.

Tensorflow for Windows

https://www.tensorflow.org/install/install_windows

Anaconda distribution includes data science modules but a minimum of the modules is installed with the miniconda version, which is is used below: https://conda.io/miniconda.html  .

Environment creation within conda

conda create -n tensor2 python=3.6

activate tensor2

pip install tensorflow

Grab the test images

wget http://download.tensorflow.org/example_images/flower_photos.tgz   

tar xfz flower_photos.tgz -C tf_files\

Setup the environment for training

set IMAGE_SIZE=224

set ARCHITECTURE="mobilenet_0.50_%IMAGE_SIZE%"

python -m scripts.retrain -h

python -m scripts.retrain   --bottleneck_dir=tf_files/bottlenecks \
  --how_many_training_steps=500   --model_dir=tf_files/models/   \
--summaries_dir=tf_files/training_summaries/"%ARCHITECTURE%" \
  --output_graph=tf_files/retrained_graph.pb  \
 --output_labels=tf_files/retrained_labels.txt  \
 --architecture="%ARCHITECTURE%"   --image_dir=tf_files/flower_photos

Label the test image or image of your own.

python -m  scripts.label_image -h

python -m scripts.label_image     --graph=tf_files/retrained_graph.pb \
      --image=915045_p30.png 

and the results are

Evaluation time (1-image): 0.464s

tulips 0.9960024
dandelion 0.002221214
sunflowers 0.0017093202
roses 5.8004178e-05
daisy 9.087755e-06

So it can be seen that at least with this model water-lilys are not in the suggested options, the style of the image makes TensorFlow to mix them to tulips…

Remember to register to the DHN18  , and pick “Miniature histories – Digitized newspapers and cultural heritage assets as source for the local history” as your workshop!

“TensorFlow for poets” command line setup  in linux

https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0

This setup has been tested e.g. with the University of Helsinki’s virtual environment, from which you can find more here.

Create virtual environment

Virtual environment is especially useful if you use virtual desktop – e.g. the tensorboard tries to install itself to common environment, which will not succeed.

virtualenv tepoet

Activate the virtual environment . Note this changes e.g. path so it uses the one under the tepoet/bin -directory

source tepoet/bin/activate

TensorFlow details:

(Skipped tensorboard due to above mentioned environment issue, but within virtualenv using tensorboard should be fine).

Install tensorflow and get the scripts from the tensorflow github.

pip install --upgrade tensorflow

git clone https://github.com/googlecodelabs/tensorflow-for-poets-2

cd tensorflow-for-poets-2/

curl http://download.tensorflow.org/example_images/flower_photos.tgz \
    | tar xz -C tf_files

ls tf_files/flower_photos/

Next thing is to to create the model and run the actual classifier to a image used in this post.

export IMAGE_SIZE=224

export ARCHITECTURE="mobilenet_0.50_${IMAGE_SIZE}"

python -m scripts.retrain -h

python -m scripts.retrain   --bottleneck_dir=tf_files/bottlenecks   \
--how_many_training_steps=500   --model_dir=tf_files/models/  \
 --summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}" \
  --output_graph=tf_files/retrained_graph.pb  \
 --output_labels=tf_files/retrained_labels.txt   \
--architecture="${ARCHITECTURE}"   --image_dir=tf_files/flower_photos

python -m  scripts.label_image -h

python -m scripts.label_image     --graph=tf_files/retrained_graph.pb \
     --image=tf_files/flower_photos/daisy/21652746_cc379e0eea_m.jpg

python -m scripts.label_image     --graph=tf_files/retrained_graph.pb\
      --image=tf_files/flower_photos/roses/2414954629_3708a1a04d.jpg

And finally run the classifier against your own image

python -m scripts.label_image     --graph=tf_files/retrained_graph.pb \
      --image=995706_kukka_1905.png

The results should be :

Evaluation time (1-image): 0.501s

roses 0.4427891
dandelion 0.36607185
sunflowers 0.1426634
tulips 0.0484466
daisy 2.9001703e-05

Remember to register to the DHN18  , and pick “Miniature histories – Digitized newspapers and cultural heritage assets as source for the local history” as your workshop!

Erilaisten tekstiaineistojen käyttöön tiedoksi Kielipankin Korp-palvelu, jossa voi hakea sisällöistä tietyillä sanoilla ja nähdä sanamuotojen esiintymät ja paikan aineistoissa. Yksinkertaisimmillaan voi valita haluamansa korpukset, jonka jälkeen hakusana ja konkordanssi-näkymään tulee tietoa sanan esiintymisestä.

Löydetystä sanasta näkee lehden tiedot josta sana löytyi, tiedot digitointiajankohdasta ja linkki digin puolelle, jos aineisto on käytettävissä.

Jos joissakin hakutuloksissa on eroa suhteessa https://digi.kansalliskirjasto.fi – palveluun, niin syinä voi olla pienet erot aineistoissa – digiin on saatettu tehdä lisäyksiä, poistoja tai  korjauksia joita Korp-palvelusta puuttuu. Lisäksi aineiston käsittelyssä ja hakutoimintojen sisäisessä toteutuksessa on eroa, joten ei tarvitse hämmästyä.

Aineistojen tutkinnassa kannattaa myös tutkia millaisia asioita hakutuloksiin tarttuu. Hakutulos ei välttämättä osu uutiseen vaan se voi osua myös esimerkiksi mainokseen tai ilmoitukseen, joka onkin voinut osua useisiin lehtiin päivä päivältä.

Olemme saaneet jo muutamia kehitysideoita kaavionäkymään, joten jos mielessäsi on jotakin, niin lähetäthän palautetta digin palaute-toiminnon avulla.

Digitalia’s and OKF’s microhistory project were happy to hear that our workshop concept was accepted to the Digital Humanities i Norden 2018 seminar. The workshop is titled: “Miniature histories – Digitized newspapers and cultural heritage assets as source for the local history” . The workshops are organized before the conference, so this session is on 6.3.2018 in Helsinki. Please reserve your calendar.

Details about the session will be shared at the beginning of next year. Do note that also workshop participants need to register themselves to the seminar.  Things we will can try hands-on, based on participant interest, for example:

• searching and creating clippings from digi.kansalliskirjasto.fi
• utilizing page texts and capturing named entities from texts (for example using Finer tool) or tag clouds
• adding data to WikiMedia with correct attributions
• utilizing WikiMedia data in other systems (API, queries)

Let me know if you would like to do something specific with the digitized newspapers or if there are local history topics, which you are interested about. All ideas are welcomed – even if we cannot do everything in the workshop, we can work with them later on. We can think about some practical application prototypes that could be of interest for researchers and all.

Happy holidays for all, and see you in next year. 🙂

Mietteitä Joulukuusesta. Lähde: Joululehti, 01.01.1887, s. 1 http://digi.kansalliskirjasto.fi/aikakausi/binding/1354895/articles/2648601?page=1 Kansalliskirjaston Digitoidut aineistot