
## TL;DR

Before publishing this post: Nothing at all.

After publishing this post: There is an article mentioning _DALL-E_ on this site.

## Hacker News - the root of all evil

There was a time when I was reading
[Hacker News](https://news.ycombinator.com/news) on regular basis but now I tend
to avoid it. Most of the time, when I was visiting it and checking all those shiny
projects and articles that people create, after a few moments of
excitement, I was ending (best case) a little melancholic about my own
achievements. So it was rather counterproductive to keep doing that. But
somehow I ended up there again last night (I was venting out some frustration
about Medium ~~comments~~ responses system in google search and one of the top
hits was HN).

And there it was - someone posted a link to _Open AI_'s
[DALL-E](https://openai.com/blog/dall-e/). And I was lost.

## DALL-E

So what is DALL-E? I will simply quote the article's introduction:

> DALL·E is a 12-billion parameter version of GPT-3 trained to generate images
> from text descriptions, using a dataset of text–image pairs. We’ve found that
> it has a diverse set of capabilities, including creating anthropomorphized
> versions of animals and objects, combining unrelated concepts in plausible
> ways, rendering text, and applying transformations to existing images.

My knowledge of generative networks never reached beyond simple GANNs and style
transfer (which is still better than my grasp on NLP). However, the article is
only 27 minutes read and not very detailed on a technical level, so it is
digestable for me. The most important thing though, are the examples of what
_DALL-E_ is capable of. It can generate:

> an illustration of a baby capybara in a wizard hat playing a guitar

and the results are great:

![DALL-E capybara example](/images/dall-e-capybara-example.png)
_Examples of DALL-E output for prompt shown above, all credit goes to Open AI_

I mean, come on! That is impressive.

Yet again, similar to the introduction
of GPT-3 architecture itself in May 2020, it makes you wonder when those models
will start replacing humans (poor me as a developer - it might seem like a joke,
but the fear is real). We should prompt some GPT-3 model to generate new
JavaScript frontend framework every week or so.

There was a vivid discussion on HN
([link here](https://news.ycombinator.com/item?id=25649557)) under the article.
Some clever points were made there. Some of them might give us, humans, some
hope for the near future. Here are those which appealed most to me:

- Some concepts probably are much more well-defined (overrepresented) in the
  training input set. And there is such possibility that those examples belong
  to those classes. And, same as the model itself, the dataset is huge, so it
  is not a surprise that you can find `anthropomorphized radish` there (search
  for `大根アニメ`, [click](https://www.google.com/search?q=%E5%A4%A7%E6%A0%B9%E3%82%A2%E3%83%8B%E3%83%A1)).
- It is a language model, so what it learns about is language (translation of
  written language to visual one in this case). It might not be as general as
  it is presented by researchers.
- The original _GPT-3_ model is 175 billion parameters (it is even hard to tell
  how _Open AI_ deploys it for training, heck even interference seems hard).
  I've read some claims that this is enough to encode around a third of the input
  data directly. With 12 billion parameters, _DALL-E_ is slightly thinner, but
  it is still big enough to be called enormous. So there are some controversies
  about reasoning capabilities of those transformer models. Do they reason or
  simply memorize the data? That would make them, very sophisticated and ML-based
  indeed, but only search algorithms (I know, exaggerating a little bit here ;) ).
- Black-box problem, which is becoming even more relevant with those models
  scaling more and more. We, humans, cannot reason about how the model works.
  We can't predict and explain why it made its decisions. It might work well in
  non-critical usages (creative work) but an unpredictable model is a no-go in
  many areas i.e. medicine or wealth management (sic!).
- Also, god only knows how processing-intensive training those transformers
  networks is, some people
  [claim](https://lambdalabs.com/blog/demystifying-gpt-3/) that for original
  _GPT-3_ it would be 355 years and a few million dollars. Better not dive into
  this topic, and not convert it to CO<sub>2</sub> tonnes equivalent.

Nevertheless, _DALL-E_ is still an IMPRESSIVE and TREMENDOUS work pushing both
natural language processing and image generation next step further.

I could probably go on and on, and try to write something about _Open AI's_
[CLIP](https://openai.com/blog/clip/) which was used in _DALL-E_'s training. Or
list some highlights about _DALL-E,_ so it doesn't seem that I am a bitter man
who can't admire the significance of this work (once again it is great). But it
is getting late (1:36 AM) so it would be probably good for me to get some sleep.

## So, what I am working on?

The whole _DALL-E_ thing was supposed to be just a short (kinda ironic) intro
from which I would move into site improvements that I am working on currently.
But again, it is getting late, so I will limit myself to listing them here:

- Added reading time (Medium like) information component to posts. But it is not
  yet populated with the correct value.
- Addition of some CSS theme mechanism to the template, so I don't need to
  remember what is hex for that gray color used in posts meta info.
- Added [hypothes.is](https://web.hypothes.is/) sidebar to the site. Found it
  accidentally during my Medium ~~comments~~ responses rage. Seems like a very
  nice concept so I've decided to give it a try.
- Read a little about SSR (funny how we, as industry, circled back to it) and web performance. Checked
  my site performance using
  [Lighthouse](https://developers.google.com/web/tools/lighthouse). Everything
  would be fine, except that adding hypothes.is sidebar gives huge performance
  penalty.

It is not rocket science nor transformer based deep learning algorithms. But
it is fun!


What have DALL-E and my site in common?

## Why was it made?

I was never good at writing nor I wanted to write. So why even bother with
personal website (for some reason I like that term better than blog)?
Well... because I can.

Maybe having a place like this will motivate me to work on some of my
side-projects (I will need something to write about, right?). Or maybe it will die like
most of those projects ;) Time will tell.

## How was it made?

I've read somewhere that it is a tradition to start a blog with a post
about building the site itself. Not sure if it is a real thing, it was some
random programmer's site (just like this one) and I never bothered to verify that.
Anyway, that will give me some volume for this first entry (though not much, things are pretty simple nowadays).

This page is git based and built with [Netlify](https://www.netlify.com/)
(lookup Jamstack if you are interested in more details).
It uses [nextjs-netlify-blog-template](https://github.com/wutali/nextjs-netlify-blog-template).

The process is as straightforward as possible:

1. Go to [nextjs-netlify-blog-template](https://github.com/wutali/nextjs-netlify-blog-template).
2. Click a "Deploy to Netlify" button (or follow this link directly to Netlify:
   [Deploy](https://app.netlify.com/start/deploy?repository=https://github.com/wutali/nextjs-netlify-blog-template&stack=cms)).
3. Connect Netlify to GitHub.
4. Provide repository name for the site and click "Save & Deploy" (template
   already configures Identity and Git Gateway services so they will be installed
   automatically).
5. Set up a domain (in my DNS provider configuration I added a `CNAME` record
   with `site` subdomain host and pointed it to my Netlify subdomain).
6. Do some cleanup of the template (remove default posts and tags).

And that is it.

Writing this paragraph took me many times longer than it took me to go through those
steps.

## Why was it made like that?

Despite working as a developer for quite some time already I had no previous
experience of creating a website all by myself. So my choices might be a little
random and whimsy. Things that led me to this decision (in random order):

* I think that server-side generated static site is a good fit for a personal
  site.
* I like Jamstack idea of git driven CMS (no backend API needed).
* As I work professionally with [Angular](https://angular.io/) lately, I
  decided that [React](https://reactjs.org/) based solution will be nice.
* I read some articles and watched some videos about
  [Next.js](https://nextjs.org/) and wanted to try it out.
* [Netlify](https://www.netlify.com/) was my first search result in Google and
  it has a free tier which should fit my needs for now.
* Medium seems like a go-to platform for blogging (or maybe it is not anymore)
  but I wanted a personal site not an account on yet another social... medium.
  Also, I might be adding something more than just blog posts (some online
  utils, apps), and it is nice to have such possibility out of the box.

I am not saying this is a perfect solution. Heck, it is probably far from optimal
or state of art. At some point I might change it to something else
(self/cloud-hosted?).

But in the end
[the point is not to catch a bunny, but to chase him, but to chase him, but to chase him](https://youtu.be/k2WtBpMkxz8).

Enjoy!

Jan

All posts / personal

What have DALL-E and my site in common?

Welcome