
## TL;DR

Before publishing this post: Nothing at all.

After publishing this post: There is an article mentioning _DALL-E_ on this site.

## Hacker News - the root of all evil

There was a time when I was reading
[Hacker News](https://news.ycombinator.com/news) on regular basis but now I tend
to avoid it. Most of the time, when I was visiting it and checking all those shiny
projects and articles that people create, after a few moments of
excitement, I was ending (best case) a little melancholic about my own
achievements. So it was rather counterproductive to keep doing that. But
somehow I ended up there again last night (I was venting out some frustration
about Medium ~~comments~~ responses system in google search and one of the top
hits was HN).

And there it was - someone posted a link to _Open AI_'s
[DALL-E](https://openai.com/blog/dall-e/). And I was lost.

## DALL-E

So what is DALL-E? I will simply quote the article's introduction:

> DALL·E is a 12-billion parameter version of GPT-3 trained to generate images
> from text descriptions, using a dataset of text–image pairs. We’ve found that
> it has a diverse set of capabilities, including creating anthropomorphized
> versions of animals and objects, combining unrelated concepts in plausible
> ways, rendering text, and applying transformations to existing images.

My knowledge of generative networks never reached beyond simple GANNs and style
transfer (which is still better than my grasp on NLP). However, the article is
only 27 minutes read and not very detailed on a technical level, so it is
digestable for me. The most important thing though, are the examples of what
_DALL-E_ is capable of. It can generate:

> an illustration of a baby capybara in a wizard hat playing a guitar

and the results are great:

![DALL-E capybara example](/images/dall-e-capybara-example.png)
_Examples of DALL-E output for prompt shown above, all credit goes to Open AI_

I mean, come on! That is impressive.

Yet again, similar to the introduction
of GPT-3 architecture itself in May 2020, it makes you wonder when those models
will start replacing humans (poor me as a developer - it might seem like a joke,
but the fear is real). We should prompt some GPT-3 model to generate new
JavaScript frontend framework every week or so.

There was a vivid discussion on HN
([link here](https://news.ycombinator.com/item?id=25649557)) under the article.
Some clever points were made there. Some of them might give us, humans, some
hope for the near future. Here are those which appealed most to me:

- Some concepts probably are much more well-defined (overrepresented) in the
  training input set. And there is such possibility that those examples belong
  to those classes. And, same as the model itself, the dataset is huge, so it
  is not a surprise that you can find `anthropomorphized radish` there (search
  for `大根アニメ`, [click](https://www.google.com/search?q=%E5%A4%A7%E6%A0%B9%E3%82%A2%E3%83%8B%E3%83%A1)).
- It is a language model, so what it learns about is language (translation of
  written language to visual one in this case). It might not be as general as
  it is presented by researchers.
- The original _GPT-3_ model is 175 billion parameters (it is even hard to tell
  how _Open AI_ deploys it for training, heck even interference seems hard).
  I've read some claims that this is enough to encode around a third of the input
  data directly. With 12 billion parameters, _DALL-E_ is slightly thinner, but
  it is still big enough to be called enormous. So there are some controversies
  about reasoning capabilities of those transformer models. Do they reason or
  simply memorize the data? That would make them, very sophisticated and ML-based
  indeed, but only search algorithms (I know, exaggerating a little bit here ;) ).
- Black-box problem, which is becoming even more relevant with those models
  scaling more and more. We, humans, cannot reason about how the model works.
  We can't predict and explain why it made its decisions. It might work well in
  non-critical usages (creative work) but an unpredictable model is a no-go in
  many areas i.e. medicine or wealth management (sic!).
- Also, god only knows how processing-intensive training those transformers
  networks is, some people
  [claim](https://lambdalabs.com/blog/demystifying-gpt-3/) that for original
  _GPT-3_ it would be 355 years and a few million dollars. Better not dive into
  this topic, and not convert it to CO<sub>2</sub> tonnes equivalent.

Nevertheless, _DALL-E_ is still an IMPRESSIVE and TREMENDOUS work pushing both
natural language processing and image generation next step further.

I could probably go on and on, and try to write something about _Open AI's_
[CLIP](https://openai.com/blog/clip/) which was used in _DALL-E_'s training. Or
list some highlights about _DALL-E,_ so it doesn't seem that I am a bitter man
who can't admire the significance of this work (once again it is great). But it
is getting late (1:36 AM) so it would be probably good for me to get some sleep.

## So, what I am working on?

The whole _DALL-E_ thing was supposed to be just a short (kinda ironic) intro
from which I would move into site improvements that I am working on currently.
But again, it is getting late, so I will limit myself to listing them here:

- Added reading time (Medium like) information component to posts. But it is not
  yet populated with the correct value.
- Addition of some CSS theme mechanism to the template, so I don't need to
  remember what is hex for that gray color used in posts meta info.
- Added [hypothes.is](https://web.hypothes.is/) sidebar to the site. Found it
  accidentally during my Medium ~~comments~~ responses rage. Seems like a very
  nice concept so I've decided to give it a try.
- Read a little about SSR (funny how we, as industry, circled back to it) and web performance. Checked
  my site performance using
  [Lighthouse](https://developers.google.com/web/tools/lighthouse). Everything
  would be fine, except that adding hypothes.is sidebar gives huge performance
  penalty.

It is not rocket science nor transformer based deep learning algorithms. But
it is fun!


All posts / nlp

What have DALL-E and my site in common?