Code it
4.6K views | +0 today
Follow
Code it
This is a curated resource for programmers and software architects. It is regularly updated with Articles, Hacks, How Tos, Examples and Code.
Curated by nrip
Your new post is loading...
Your new post is loading...
Scooped by nrip
Scoop.it!

How to Train and Test an AI Language Translation System

How to Train and Test an AI Language Translation System | Code it | Scoop.it
In the previous article, we built a deep learning-based model for automatic translation from English to Russian. In this article, we’ll train and test this model.
 
Here we'll create a Keras tokenizer that will build an internal vocabulary out of the words found in the parallel corpus, use a Jupyter notebook to train and test our model, and try running our model with self-attention enabled.
 
 
No comment yet.
Scooped by nrip
Scoop.it!

Tools for Building AI Language Translation Systems

Tools for Building AI Language Translation Systems | Code it | Scoop.it

Google Translate works so well, it often seems like magic. But it’s not magic — it’s deep learning!

 

In this series of articles, we’ll show you how to use deep learning to create an automatic translation system. This series can be viewed as a step-by-step tutorial that helps you understand and build a neuronal machine translation.

 

This series assumes that you are familiar with the concepts of machine learning: model training, supervised learning, neural networks, as well as artificial neurons, layers, and backpropagation.

 

In this article, we’ll examine the tools we'll need to use to build an AI language translator.

 

Multiple frameworks provide APIs for deep learning (DL). The TensorFlow + Keras combination is by far the most popular, but competing frameworks, such as PyTorch, Caffe, and Theano, are also widely used.

 

These frameworks often practice the black box approach to neural networks (NNs) as they perform most of their "magic" without requiring you to code the NN logic. There are other ways to build NNs — for instance, with deep learning compilers.

 

The following table lists the versions of the Python modules we’ll use. All these modules can be explicitly installed using the ==[version] flag at the end of a pip command. For instance: "pip install tensorflow==2.0".

 

The code we're writing should work on any operating system but note that we're using Python 3, so make sure you have it installed. If your system has both Python 2 and Python 3 installed, you'll need to run pip3 instead of pip in the install commands below:

 

 

module version TensorFlow 2.3.1 Keras 2.1.0 numpy 1.18.1 pandas 1.1.3 word2vec

0.11.1

 

read the entire article for the list of instructions at https://www.codeproject.com/Articles/5299747/Tools-for-Building-AI-Language-Translation-Systems

 

No comment yet.