Pull to refresh
356.55

Python *

Interpreted high-level programming language for general-purpose programming

Show first
Rating limit
Level of difficulty

Building a GPT-like Model from Scratch with Detailed Theory and Code Implementation

Reading time 14 min
Views 33K

Unlock the power of Transformer Neural Networks and learn how to build your own GPT-like model from scratch. In this in-depth guide, we will delve into the theory and provide a step-by-step code implementation to help you create your own miniGPT model. The final code is only 400 lines and works on both CPUs as well as on the GPUs. If you want to jump straight to the implementation here is the GitHub repo.

Transformers are revolutionizing the world of artificial intelligence. This simple, but very powerful neural network architecture, introduced in 2017, has quickly become the go-to choice for natural language processing, generative AI, and more. With the help of transformers, we've seen the creation of cutting-edge AI products like BERT, GPT-x, DALL-E, and AlphaFold, which are changing the way we interact with language and solve complex problems like protein folding. And the exciting possibilities don't stop there - transformers are also making waves in the field of computer vision with the advent of Vision Transformers.

Read more
Total votes 25: ↑25 and ↓0 +25
Comments 1

Tips and tricks from my Telegram-channel @pythonetc, December 2019

Reading time 2 min
Views 1.5K


It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Previous publications.


Different asyncio tasks obviously have different stacks. You can view at all of them at any moment using asyncio.all_tasks() to get all currently running tasks and task.get_stack() to get a stack for each task.
Read more →
Total votes 27: ↑26 and ↓1 +25
Comments 0

Tips and tricks from my Telegram-channel @pythonetc, November 2019

Reading time 3 min
Views 2.6K

Tips and tricks from my Telegram-channel @pythonetc, November 2019

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Previous publications.



PATH is an environment variable that stores paths where executables are looked for. When you ask your shell to run ls, the shell looks for the ls executable file across all paths that are presented in PATH.
Read more →
Total votes 31: ↑30 and ↓1 +29
Comments 1

Faster ENUM

Reading time 9 min
Views 2.2K

tl;dr


github.com/QratorLabs/fastenum
pip install fast-enum

What are enums


(If you think you know that — scroll down to the “Enums in Standard Library” section).

Imagine that you need to describe a set of all possible states for the entities in your database model. You'll probably use a bunch of constants defined as module-level attributes:
# /path/to/package/static.py:
INITIAL = 0
PROCESSING = 1
PROCESSED = 2
DECLINED = 3
RETURNED = 4
...

...or as class-level attributes defined in their own class:
class MyModelStates:
  INITIAL = 0
  PROCESSING = 1
  PROCESSED = 2
  DECLINED = 3
  RETURNED = 4

That helps you refer to those states by their mnemonic names, while they persist in your storage as simple integers. By this, you get rid of magic numbers scattered through your code and make it more readable and self-descriptive.

But, both the module-level constant and the class with the static attributes suffer from the inherent nature of python objects: they are all mutable. You may accidentally assign a value to your constant at runtime, and that is a mess to debug and rollback your broken entities. So, you might want to make your set of constants immutable, which means both the number of constants declared and the values they are mapped to must not be modified at runtime.
Read more →
Total votes 28: ↑28 and ↓0 +28
Comments 0

Tips and tricks from my Telegram-channel @pythonetc, October 2019

Reading time 3 min
Views 1K

It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Previous publications


If you want to iterate over several iterables at once, you can use the zip function (it has nothing to do with ZIP file format):
Read more →
Total votes 29: ↑28 and ↓1 +27
Comments 0

Tips and tricks from my Telegram-channel @pythonetc, August 2019

Reading time 4 min
Views 1.6K


It is a new selection of tips and tricks about Python and programming from my Telegram-channel @pythonetc.

Previous publications


If an instance of a class doesn’t have an attribute with the given name, it tries to access the class attribute with the same name.

>>> class A:
...     x = 2
...
>>> A.x
2
>>> A().x
2
Read more →
Total votes 29: ↑27 and ↓2 +25
Comments 1

Contextual Emotion Detection in Textual Conversations Using Neural Networks

Reading time 10 min
Views 3.7K

Nowadays, talking to conversational agents is becoming a daily routine, and it is crucial for dialogue systems to generate responses as human-like as possible. As one of the main aspects, primary attention should be given to providing emotionally aware responses to users. In this article, we are going to describe the recurrent neural network architecture for emotion detection in textual conversations, that participated in SemEval-2019 Task 3 “EmoContext”, that is, an annual workshop on semantic evaluation. The task objective is to classify emotion (i.e. happy, sad, angry, and others) in a 3-turn conversational data set.
Read more →
Total votes 37: ↑37 and ↓0 +37
Comments 0

Google News and Leo Tolstoy: visualizing Word2Vec word embeddings using t-SNE

Reading time 7 min
Views 13K

Everyone uniquely perceives texts, regardless of whether this person reads news on the Internet or world-known classic novels. This also applies to a variety of algorithms and machine learning techniques, which understand texts in a more mathematical way, namely, using high-dimensional vector space.

This article is devoted to visualizing high-dimensional Word2Vec word embeddings using t-SNE. The visualization can be useful to understand how Word2Vec works and how to interpret relations between vectors captured from your texts before using them in neural networks or other machine learning algorithms. As training data, we will use articles from Google News and classical literary works by Leo Tolstoy, the Russian writer who is regarded as one of the greatest authors of all time.

We go through the brief overview of t-SNE algorithm, then move to word embeddings calculation using Word2Vec, and finally, proceed to word vectors visualization with t-SNE in 2D and 3D space. We will write our scripts in Python using Jupyter Notebook.

Read more →
Total votes 28: ↑28 and ↓0 +28
Comments 0

Making a DIY thermal camera based on a Raspberry Pi

Reading time 6 min
Views 60K
image

Hi everyone!

Winter has arrived, and so I had to check the thermal insulation of my out of town residence dacha. And it just turned out a famous Chinese marketplace started to sell cheap thermal camera modules. So I decided to DIY it up and build a rather exotic and useful thing — a heat visor for the home. Why not? Especially since I had a Raspberry Pi lying around anyway… The result is down below.
Read more →
Total votes 25: ↑25 and ↓0 +25
Comments 0

Authors' contribution