Title: Failure. Or: why Rust is probably the best programming language ever created
Category: Blog
Tags: Dev Diary, rust, programming, Reflections
Category: OldBlog
Tags: Dev Diary, Rust, Programming, Reflections
Date: 2017-12-20
*This post is two stories.* One is about accepting and recognising personal failure, reflecting and growing from it; the other is about an incredibly and seemingly endlessly powerful programming language, called *Rust*.
**This post is two stories**. One is about accepting and recognising personal failure, reflecting and growing from it; the other is about an incredibly and seemingly endlessly powerful programming language, called *Rust*. Let's begin...
**In the summer of 2014** I started a project which was kind of insane. I knew it was insane, yet I embarked on that journey regardless. I wanted to write a password manager. I chose Ruby as a language because I didn't know many others and was – in more than one way – still a programmer novice.
In the summer of 2014 I started a project which was kinda insane. I knew it was insane, yet I embarked on that journey. I wanted to write a password manager, because the ones available on Linux (which I had just switched to) didn't satisfy me. I chose Ruby as a language because I didn't know many others and was, in many ways, still a programmer novice.
The details of development aren't too important. About 6-8 months into the project I had written something rather cool and functional. It wasn't very fast, the code base was a bit of a mess and I was having issues with packaging. But, at the core, I really liked what I had made, which had shifted from just being a password manager to being a universal, platform-independant secrets manager, close to a keychain. In my mind applications could write sensitive information into a "vault" which was managed by this project, without having to worry too much about access rights, authentication or anything else.
The details of this development aren't too important. I had written something rather cool and functional within maybe 6-8 months of development. It wasn't very fast, the code base was a bit of a mess and I was having issues with packaging. But, at the core, I really liked what I had made, which had shifted from just being a password manager to being basically a universal, platform-independant secrets manager, close to a keychain. In my mind applications could write sensitive information into a "vault" which was managed by this project, without having to worry too much about access rights, authentication or anything else.
## So far so good; this is how both stories start.
**That's the beginning of both stories.**
Over the next few years this project would take me over and, ultimately, destroy me. I had gotten it into my mind that the cryptography should have been handled by something more low-level, something more "advanced". I talked to people, I looked at languages and in the end, thinking I had more experience now, chose C++ to re-write the project in. *This was the beginning of the end.* It took me another six months to get the basics done, getting caught up on nitty gritty details.
Over the next few years this project would take me over and, ultimately, drive me insane. I had gotten it into my mind that the cryptography should have been handled by something more low-level, something more "advanced". I talked to people, I looked at languages and in the end, thinking I had more experience now, chose C++ to re-write the project in. This was the beginning of the end. It took me months to get the basics done, getting caught up on nitty gritty details. I ended up switching to C, back to C++, then back to C again, not being satisfied with the way that one or the other language handled things. And the scope was out of control. I didn't want to make a cute little secrets manager anymore. I wanted to make a database. It had transactions, sharding, multi-user access, backups, countless optimisations, run modes and even it's own SQL-like qeury language. I went completely insane and lost all grasp of what it was I wanted to create. With time, re-writing the same parts of the code again and again, creating new libraries to handle even smaller tasks that had been completely trivial in Ruby, I stopped.
I ended up switching to C, back to C++, *then back to C again*, not being satisfied with the way that one or the other language handled things. And the scope was out of control. I didn't want to make a cute little secrets manager anymore. I wanted to make a database. It had transactions, sharding, multi-user access, backups, countless optimisations, run modes and even it's own SQL-like query language. I went completely overboard and lost all grasp of what it was I wanted to create. After literally years, re-writing the same parts of the code again and again, creating new libraries to handle even smaller tasks that had been completely trivial in Ruby, **I stopped.**
What this project had turned into wasn't maintainable anymore. It didn't even really make any sense. It had no use-case outside of "being cool" and that wasn't really enough to motivate me anymore to work on it. I was also caught up with other work, getting involved with the Google Summer of Code 2016, then slowly fading work on the project into the background. This wasn't a concious decision though. In my mind, I was just putting it on hold, learning from all of my failures and then trying myself again at another re-write. I didn't know that *not* trying again would be the act of learning from it.
What this project had turned into wasn't maintainable. It didn't even really make any sense. It had no use-case, besides "being cool" and that wasn't really enough to motivate me anymore. I was also caught up with other work, getting involved with the Google Summer of Code 2016, then slowly fading work on the project into the background. This wasn't a conscious decision though. In my mind, I was just putting it on hold, learning from all of my failures and then trying myself again at another re-write. I didn't know that *not* trying again would be the act of learning from it.
As hackers, we're often compelled to take on the world. Everything seems plausible, sometimes trivial. We understand technology in a way that most people don't and in that, we gain confidence in our abilities past the point of reality. We want to make things, break things, modify things. And we forget our own limitations, time and scope. We end up starting so many things that we never finish. Or we get obsessed with something that doesn't make any sense.
<br/>
It took me over 2 years to understand that I can't let my impulse to adventure drive the way I work. I love open source and I love working on things that are just *free* and out in the open. I want to help build an ecosystem of tools and applications that help people, without any cost or baggage of being for a closed down system. But learning, that there were things that I can't do, that maybe the way that I viewed work, problems and how to tackle them was *fallable*, that took some more time to understand. In the end, everything I did on this project was a collosal waste of time. It's still on my github, more as a reminder to myself of how failure works.
As hackers, we are often compelled to take on the world. Everything seems plausible, sometimes trivial. We understand technology in a way that most people don't and in that, we gain confidence in our abilities past the point of reality. Hubris. We want to make things, break things, modify things. And we forget our own limitations, time and scope. We end up starting so many things that we never finish. Or we get obsessed with something that doesn't make any sense.
It has nothing to do with not knowing how to solve a problem. It has nothing to do with failing to understand code or a language or a toolkit... It has *everything* to do with not knowing how to limit a project, and *when to stop*...
<br/>
*This is the end of story one.*
It took me over 2 years to understand that I can't let my impulse to adventure drive the way I work. I love open source and I love working on things that are just *free* and out in the open. I want to help build an ecosystem of tools and applications that help people, without any cost or baggage of being for a closed down system. But learning, that there were things that I can't do, that maybe the way that I viewed work, problems and how to tackle them was *fallable*, that took some more time to understand. In the end, everything I did on this project was a collosal waste of time. It's still on my github, more as a reminder to myself of how failure works...
It has nothing to do with not knowing how to solve a problem. It has nothing to do with failing to understand code or a language or a toolkit... It has *everything* to do with not knowing how to limit a project, **and when to stop...**
## This is the end of story one
It's been nearly a year since I worked on this project (or the 5th iteration of a re-write anyways), in the meantime I've worked on many small things, trying to keep in mind what I want to do, what is plausible and also useful. And in the meantime I've come into contact with a magical programming language: *Rust*!
@ -40,6 +45,12 @@ I remembered what I had thought about before. Limitation of scope, accepting lim
Rust makes it incredibly simple to do rapid prototyping. Yes, the language is very strict sometimes. Yet it has this feeling of "throwing shit against the wall" and seeing what sticks. With the added benefit that there are compile-time checks that make sure that there are no serious issues with your program. You can still write bad code, it just seriously limits the damage you can do. And that makes it incredibly fun to write with.
What's my point with all of this. Well, first that I love Rust :P But secondly that sometimes our failure isn't just our own shortcomings but the failure of selecting the right tools for the job. And maybe, this is something we should foster a lot more in our community. Rust is an amazing language for many things but it also has it's limits. There are many people in the hacker culture who stick to their technologies because they feel familiar, dragging others into their little bubbles because they want to expand their influence, never considering if what they're advocating is sensible or scalable.
## What's the point of all this?
Well, first that I love Rust 😝.
But secondly that sometimes failure looks different than what we might expect. It's not about failing on a technical but either on a social or planning level. And maybe that this is something we should talk about and foster in the hacker community.
Rust is an amazing language for many things but it also has it's limits. There are countless people in the hacker culture who stick to their technologies because they feel familiar, dragging others into their little bubbles because they want to expand their influence, never considering if what they're advocating is sensible or scalable.
And in the end, shouldn't we all strife to learn new things, broaden our horizons and, last but not least, chose the right tool, for the right job.
This isn't just something we (as a culture) do with tools, it also happens on a social level. And in the end, shouldn't we all strife to learn new things, broaden our horizons and, last but not least, choose the right tool, for the right job?
**Let me tell you an even more factual statement**
*UI programming in LibGDX is even more terrible*
I am a big fan of LibGDX. It's a really nifty library/ framework to get started with game development if you're more comfortable inside a code editor than a full blown game engine that is more targeted towards designers and artists. And I put my money where my mouth is: I have a series about LibGDX development for beginners on this blog and work almost exclusively with it when it comes to my own projects.
Yet, there is something that bothers me and there didn't seem to be a great solution to fix it. UI code structure. In this post I want to highlight a utility I have written for LibGDX which is very easily embeddable into your existing projects which will you help structure UI code more efficiently.
## The root problem
The reason I dislike UI programming with LibGDX is that it usually results in very long code files or passing dozens of parameters into sub-classes that are needed to update the UI for button presses, etc.
This goes so far that I have written an editor for game assets before just to realise that (once the development was complete) it had become completely unmaintainable and I had to start from scratch with better structure. It is incredibly easy to just throw out a UI design with Scene2D and LibGDX but unfortunately it is equally easy to produce very bad code which will turn into a big spaghetti mess.
Let's look at an example problem that I wanted to solve.
Looking at this structure we have three main components that interact with each other. We have a class that handles UI logic (setting up actors in tables, adding listeners, etc), we have a window state which in the particular case which made me write an alternative was a "Lobby Handle" which coordinated what players were going to enter a match, the map, game mode and if everybody in the multiplayer match was set to "Ready". Lastly we have the actual network signal handlers that listen to TCP/ UDP packets and execute code to write/ read from the window state as well as update UI elements.
Implementing this structure with Scene2D and LibGDX will result in a lot of very ugly code. Because the network signals need to know everything about the UI (how it is structured, etc). And our window state can be written to by two different sources which means that we need to mutex it to avoid race conditions.
So, what was I trying to solve? First a bit of limitation of scope. Because a lot of UI problems have been solved over and over again and usually at the cost of runtime performance or with a *lot* of extra code.
1. UI code doesn't have to be embedded in a screen
2. All UI code can access the shared context of the screen
3. UI elements can update each other
4. Clean API that can be called on from anywhere (with a reference to the handle) that triggers range of functions.
So with that in mind, this is what I did.
```java
class MyUIHandle extands UIHandle {
public static enum UI implements UI_BASE {
PLAYER_LIST;
}
{ /** Initialiser block for new objects */
registerHandle(new PlayerList(), UI.PLAYER_LIST);
// ... more handles
}
@Override
public void initialise(Stage s, Object ... var) { ... }
public class PlayerList extends UIContainer {
@Override
public void initialise(Stage s) { ... }
// Define more API here ...
}
}
```
When we initialise a new `UIHandle` the initialiser block will create our `PlayerLists` and register them with the `UIHandle`. That code is hidden away from you. You can see that we're implementing a different enum type that we overload with values so that we can address submodules via a compile-time checkable value (such as enums). From inside (and outside) this class `UIContainer's` are available via `handle.get(UI.SUB_HANDLE)`. Obviously keeping your enum labels short will make your function calls snappier :)
The following graphic will sort-of explain the layout in more detail.
What you might also notice is that the `UIHandle` has an initialise function with variadic parameters while the `UIContainer` class only takes a stage. That is because window context is stored once in the `UIHandle` and then accessable from all `UIContainer` classes. This way we only need to do the inversion of control pattern once instead of for every sub-component.
You can keep the `UIContainer` classes outside this code-file. Then you might however want to provide a construct that does another inversion of control so that an external `UIContainer` can access the context provided via initialise!
```java
public class PlayerList extends UIContainer {
private MyUIHandle parent;
public PlayerList(MyUIHandle parent) { this.parent = parent; }
// ...
}
```
Now let's talk about that public API. In our original example we wanted to have networking code update some UI elements. And we want UI elements to update other UI elements. So first of all, we keep context in each `UIContainer` about what UI elements are accessable to it. So what we can do in every of our submodules is this:
It also means that if we get new data from – say – a network socket or AI simulation, we can very easily update data in some random UI element.
```java
handle.get(UI.PLAYER_LIST).populate(playerList);
```
So all in all, we have solved the following problems:
1. We have access to all game state in the UI code without passing too many parameters into lots of sub-classes
2. UI code can be moved into lots of files for easier understandability
3. Context isn't duplicated
4. UI code can update other UI code without needing a direct reference to it.
The individual `UIContainer` instances are essentially independant of each other via dependency injection.
This library isn't done yet. Most of this is kinda hacked together to fit into **my** game. But I'm interested in making it more generic and putting it on Github. Especially because I can see myself using it again in the future.
Hope this might be useful to somebody out there. If you have questions, comments, hatemail...
[Twitter](https://twitter.com/spacekookie) or [E-Mail](mailto:kookie@spacekookie.de)
Recently I've started learning/ using Moonscript. It's a language that compiles to [lua](https://www.lua.org/) and as such can run in the LuaJIT, an alternative lua engine which allows very easy and *fast* ffi calls into native code. This makes lua code capable of writing very performant applications and games that use native rendering, window creation or general libraries.
But in my opinion lua has always felt a bit cumbersome. I use awesomewm so I had to write it occasionally to customise my UI layout. And this is where Moonscript comes in. It's a lot of syntactic sugar on top of lua as well as some other concepts such as object orientation which lua just plain out doesn't have. And while yes, you can write good code without OO (*cough* **C***cough*) it is a nice tool to have in your pocket, especially when writing GUI applications or games.
## The language
```Moonscript
class Thing
name: "unknown"
class Person extends Thing
say_name: => print "Hello, I am #{@name}!"
with Person!
.name = "MoonScript"
\say_name!
```
As you can see Moonscript is an indentation based language which (in my opinion) combines syntactic elements from lua and ruby together. In the snippet above (which is from the [moonscript website](http://moonscript.org/)) you can see classes, inheritance as well as the `with` keyword which allows you to initialise/ work with objects without typing it's variable name over and over again.
If you want to learn more about the language, I can only recommend you have a look at the [Moonscript in 15 minutes guide](https://github.com/leafo/moonscript/wiki/Learn-MoonScript-in-15-Minutes)
## How to use it
You can just write Moonscript files, add `#!/usr/bin/env moon` to them and get going. Obviously that's pretty cool for little scripts that you just want to get going. But not so great for larger applications because a) you don't have access to `ffi` via luaJIT and b) it adds additional startup cost.
So instead for my projects so far (which so far are a [game](https://github.com/spacekookie/dinodino) and a desktop app) I use a `Makefile` to build and run the Moonscript compiler and then execute the `init.lua` with luajit.
So...I'm kinda excited about this. Most of the code I write is either in C or Java (depending on what exactly I'm doing). And those two strongly typed and compiled languages have served me well and will continue to be my go-to solutions for a lot of problems.
But I've long been looking for a dynamicly typed, interpreted/ just-in-time compiled language that I can use for anything from little scripts to medium-sized desktop applications. I used to use python for this but have recently (over the last 6-9 months) fallen out of love and developed a rather passionate dislike of it and it's ecosystem.
My current project will get it's own little article at some point but I don't mind teasing the progress here. I'm writing a new UI for redshift which works with X11 linux backends and is heavily inspired by f.lux on MacOS. It's written in moonscript, with my own forked version of redshift (which I call [libredshift](https://github.com/spacekookie/libredshift)). It's on [github](https://github.com/spacekookie/redshift_ctrl) and licensed under MIT.
Hope I've made you a little curious about Moonscript :)
It's winter, rebuilding my website is a tradition...right? **Happy new year everybody.**
This has been a long time coming. I've not really been happy with the way my website looked for a while and have been playing around with new designs for the past few months. I also took that opportunity to throw out a few old articles, fix formatting on others and generally do house-keeping.
The whole thing is still using [Pelican](https://blog.getpelican.com/) to generate pages but now with a completely new theme and new plugins 🎉
This re-design also decreases complexity. The old theme was massively too complicated and I've now taken it down to 3 (or 4?) templates. Working around the old theme and what Pelican expected was an interesting experience. Especially since it felt less like building a website and more like working with a game engine where small changes make lots of magic happen and *voilá*.
There are a few things I want to write about...*soon*. Until then, there is an easteregg hidden somewhere. Let's see who finds it first 😉
