Machine learning *

The basis of artificial intelligence

Articles Posts News Authors

bashnick Jan 24 2023 at 23:03

Building a GPT-like Model from Scratch with Detailed Theory and Code Implementation

14 min

34K

Open Data Science corporate blogPython*Machine learning*Artificial IntelligenceNatural Language Processing*

Tutorial

Unlock the power of Transformer Neural Networks and learn how to build your own GPT-like model from scratch. In this in-depth guide, we will delve into the theory and provide a step-by-step code implementation to help you create your own miniGPT model. The final code is only 400 lines and works on both CPUs as well as on the GPUs. If you want to jump straight to the implementation here is the GitHub repo.

Transformers are revolutionizing the world of artificial intelligence. This simple, but very powerful neural network architecture, introduced in 2017, has quickly become the go-to choice for natural language processing, generative AI, and more. With the help of transformers, we've seen the creation of cutting-edge AI products like BERT, GPT-x, DALL-E, and AlphaFold, which are changing the way we interact with language and solve complex problems like protein folding. And the exciting possibilities don't stop there - transformers are also making waves in the field of computer vision with the advent of Vision Transformers.

+25

andrewbo29 Sep 18 2019 at 14:22

How we made landmark recognition in Cloud Mail.ru, and why

11 min

2.4K

VK corporate blogAlgorithms*Image processing*Machine learning*Artificial Intelligence

With the advent of mobile phones with high-quality cameras, we started making more and more pictures and videos of bright and memorable moments in our lives. Many of us have photo archives that extend back over decades and comprise thousands of pictures which makes them increasingly difficult to navigate through. Just remember how long it took to find a picture of interest just a few years ago.

One of Mail.ru Cloud’s objectives is to provide the handiest means for accessing and searching your own photo and video archives. For this purpose, we at Mail.ru Computer Vision Team have created and implemented systems for smart image processing: search by object, by scene, by face, etc. Another spectacular technology is landmark recognition. Today, I am going to tell you how we made this a reality using Deep Learning.

+43

sismetanin Aug 1 2019 at 10:35

Contextual Emotion Detection in Textual Conversations Using Neural Networks

10 min

3.7K

VK corporate blogPython*Data Mining*Big Data*Machine learning*

Nowadays, talking to conversational agents is becoming a daily routine, and it is crucial for dialogue systems to generate responses as human-like as possible. As one of the main aspects, primary attention should be given to providing emotionally aware responses to users. In this article, we are going to describe the recurrent neural network architecture for emotion detection in textual conversations, that participated in SemEval-2019 Task 3 “EmoContext”, that is, an annual workshop on semantic evaluation. The task objective is to classify emotion (i.e. happy, sad, angry, and others) in a 3-turn conversational data set.

+37

kitashov Jul 12 2019 at 08:11

AI-Based Photo Restoration

7 min

18K

VK corporate blogAlgorithms*Image processing*Machine learning*

Hi everybody! I’m a research engineer at the Mail.ru Group computer vision team. In this article, I’m going to tell a story of how we’ve created AI-based photo restoration project for old military photos. What is «photo restoration»? It consists of three steps:

we find all the image defects: fractures, scuffs, holes;
we inpaint the discovered defects, based on the pixel values around them;
we colorize the image.

Further, I’ll describe every step of photo restoration and tell you how we got our data, what nets we trained, what we accomplished, and what mistakes we made.

+32

sismetanin Apr 30 2019 at 08:42

Google News and Leo Tolstoy: visualizing Word2Vec word embeddings using t-SNE

7 min

13K

VK corporate blogPython*Big Data*Data visualization*Machine learning*

Everyone uniquely perceives texts, regardless of whether this person reads news on the Internet or world-known classic novels. This also applies to a variety of algorithms and machine learning techniques, which understand texts in a more mathematical way, namely, using high-dimensional vector space.

This article is devoted to visualizing high-dimensional Word2Vec word embeddings using t-SNE. The visualization can be useful to understand how Word2Vec works and how to interpret relations between vectors captured from your texts before using them in neural networks or other machine learning algorithms. As training data, we will use articles from Google News and classical literary works by Leo Tolstoy, the Russian writer who is regarded as one of the greatest authors of all time.

We go through the brief overview of t-SNE algorithm, then move to word embeddings calculation using Word2Vec, and finally, proceed to word vectors visualization with t-SNE in 2D and 3D space. We will write our scripts in Python using Jupyter Notebook.

+28

cointegrated Oct 21 2018 at 08:12

How linear algebra is applied in machine learning

5 min

14K

Mathematics*Machine learning*

When you study an abstract subject like linear algebra, you may wonder: why do you need all these vectors and matrices? How are you going to apply all this inversions, transpositions, eigenvector and eigenvalues for practical purposes?

Well, if you study linear algebra with the purpose of doing machine learning, this is the answer for you.

In brief, you can use linear algebra for machine learning on 3 different levels:

application of a model to data;
training the model;
understanding how it works or why it does not work.

+25