package tensorflow

README.md

The tensorflow-ocaml project provides some OCaml bindings for TensorFlow.

Experimental ocaml bindings for PyTorch can be found in the ocaml-torch repo.

Installation

Use opam to install the tensorflow-ocaml package. This version currently lags significantly behind the GitHub tip.

opam install tensorflow

Get the TensorFlow Library

The opam package starting from version 0.0.8 requires the version 1.0 of the TensorFlow library to be installed on your system under the name libtensorflow.so. The current github tip requires 1.10.0. Two possible ways to obtain it are:

• Build the library from source. Perform the following steps:

  1. Install the Bazel build system.

  2. Clone the TensorFlow repo:

     git clone --recurse-submodules -b r1.10 https://github.com/tensorflow/tensorflow

  3. Configure the build (you will be asked if you want to enable CUDA support):

     cd tensorflow/
     ./configure
     

  4. Compile the library:

     bazel build -c opt tensorflow:libtensorflow.so

     The binary should appear under bazel-bin/tensorflow/libtensorflow.so.

• Use prebuilt binaries from Google. The releases are available for download in URLs of the form: https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-TYPE-OS-ARCH-VERSION.tar.gz. For example:

  • CPU-only, Linux, x86_64. [1.0.0] [1.10.0]

  • GPU-enabled, Linux, x86_64. [1.0.0] [1.10.0]

  • CPU-only, OS X, x86_64. [1.0.0] [1.10.0]

  • GPU-enabled, OS X, x86_64. [1.0.0] [1.10.0]

Once you have obtained the library, you should install it system-wide, or set the LIBTENSORFLOW environment variable.

export LIBTENSORFLOW={path_to_folder_with_libtensorflow.so}

Build a simple example or run utop

Download a very simple example and compile it with the following command:

ocamlbuild forty_two.native -use-ocamlfind -pkg tensorflow -tag thread

Then run it via ./forty_two.native. You should now be all set up, enjoy!

You can also run a Tensorflow enabled utop via: make utop.

Examples

Tensorflow-ocaml includes two different APIs to write graphs.

Using the Graph API

The graph API is very close to the original TensorFlow API.

• Some MNIST based tutorials are available in the examples directory. A simple Convolutional Neural Network can be defined as follows:

  let ys_ =
    O.Placeholder.to_node xs
    |> Layer.reshape ~shape:[ -1; 28; 28; 1 ]
    |> Layer.conv2d ~ksize:(5, 5) ~strides:(1, 1) ~output_dim:32
    |> Layer.max_pool ~ksize:(2, 2) ~strides:(2, 2)
    |> Layer.conv2d ~ksize:(5, 5) ~strides:(1, 1) ~output_dim:64
    |> Layer.max_pool ~ksize:(2, 2) ~strides:(2, 2)
    |> Layer.flatten
    |> Layer.linear ~output_dim:1024 ~activation:Relu
    |> O.dropout ~keep_prob:(O.Placeholder.to_node keep_prob)
    |> Layer.linear ~output_dim:10 ~activation:Softmax
  in
  

• examples/load/load.ml contains a simple example where the TensorFlow graph is loaded from a file (this graph has been generated by examples/load.py),

• examples/basics contains some curve fitting examples. You will need gnuplot to be installed via opam to run the gnuplot versions.

Using the FNN API

The FNN API is a layer based API to easily build neural-networks. A linear classifier could be defined and trained in a couple lines:

  let input, input_id = Fnn.input ~shape:(D1 image_dim) in
  let model =
    Fnn.dense label_count input
    |> Fnn.softmax
    |> Fnn.Model.create Float
  in
  Fnn.Model.fit model
    ~loss:(Fnn.Loss.cross_entropy `mean)
    ~optimizer:(Fnn.Optimizer.gradient_descent ~learning_rate:8.)
    ~epochs
    ~input_id
    ~xs:train_images
    ~ys:train_labels;

A complete VGG-19 model can be defined as follows:

let vgg19 () =
  let block iter ~block_idx ~out_channels x =
    List.init iter ~f:Fn.id
    |> List.fold ~init:x ~f:(fun acc idx ->
      Fnn.conv2d () acc
        ~name:(sprintf "conv%d_%d" block_idx (idx+1))
        ~w_init:(`normal 0.1) ~filter:(3, 3) ~strides:(1, 1) ~padding:`same ~out_channels
      |> Fnn.relu)
    |> Fnn.max_pool ~filter:(2, 2) ~strides:(2, 2) ~padding:`same
  in
  let input, input_id = Fnn.input ~shape:(D3 (img_size, img_size, 3)) in
  let model =
    Fnn.reshape input ~shape:(D3 (img_size, img_size, 3))
    |> block 2 ~block_idx:1 ~out_channels:64
    |> block 2 ~block_idx:2 ~out_channels:128
    |> block 4 ~block_idx:3 ~out_channels:256
    |> block 4 ~block_idx:4 ~out_channels:512
    |> block 4 ~block_idx:5 ~out_channels:512
    |> Fnn.flatten
    |> Fnn.dense ~name:"fc6" ~w_init:(`normal 0.1) 4096
    |> Fnn.relu
    |> Fnn.dense ~name:"fc7" ~w_init:(`normal 0.1) 4096
    |> Fnn.relu
    |> Fnn.dense ~name:"fc8" ~w_init:(`normal 0.1) 1000
    |> Fnn.softmax
    |> Fnn.Model.create Float
  in
  input_id, model

This model is used in the following example to classify an input image. In order to use it you will have to download the pre-trained weights.

There are also some MNIST based examples.

Other Examples

The examples directory contains various models among which:

• A simplified version of char-rnn illustrating character level language modeling using Recurrent Neural Networks.

• Neural Style Transfer applies the style of an image to the content of another image. This uses some deep Convolutional Neural Network.

• Some variants of Generative Adverserial Networks. These are used to generate MNIST like images.

Dependencies

• dune is used as a build system.

• ocaml-ctypes is used for the C bindings.

• Base is only necessary when generating the TensorFlow graph from OCaml, the wrapper itself does not need it.

• The code in the piqi directory comes from the Piqi project. There is no need to install piqi though.

• Cmdliner is used for command line interfaces.

• Gnuplot-ocaml is an optional dependency used by a couple examples.

• npy-ocaml is used to read/write from npy/npz files.

• camlimages handles loading and writing image files in jpeg/png format.