Poki Nanpa: A Toki Pona programming language

There is a standardised way to turn a specification of a language into a parser that can take program code and turn it into something that can be executed by a program. The two main steps are tokenisation into a list of tokens, followed by parsing into an “abstract syntax tree”; a data structure representing the shape of a program.

After beginning by trying to invent a parser that used string operations to try to build an abstract syntax tree from scratch, I instead switched to following a more standard method, outlined in this helpful guide. This involves a parser that “eats” tokens one by one according to what is expected based on what kind of language feature the parser thinks it is currently parsing. If it encounters an unexpected token, it throws a syntax error.

I am implementing this parser in JavaScript as part of the web interface I started working on, but the idea of a specification is that someone else would be able to come along and implement Toki Nanpa in any other environment.

Since Poki Nanpa is a functional language, everything happens at once. That puts it at odds with most graphical programs, which require a sequence of instructions. The solution is to use the functional aspects of Poki Nanpa to construct a list of instructions, which is then executed in sequence. To pull that off, it needs to be able to handle lists.

In the specification, I have outlined the list operations that will be available in Poki Nanpa. These include adding and removing elements, mapping one list to another using a defined function, and passing the elements of lists as parameters into functions. That also meant coming up with a way to specify whether a function is meant to be evaluated or passed when it is used as a parameter. Right now I’m not sure if the solution I’ve come up with is airtight, which is why it’s time to start implementing the interpreter and finding the gaps and loopholes in the specification.

With the language specification nearly finished, it’s time to put together a very basic prototype of what the actual interface for programming in Poki Nanpa is going to look like so that I can start writing the interpreter in an environment where it can be tested. Working from the mock-up I made for the first deliverable, I have created a web page that can take input from a text area, run it through a parser, and output something to a canvas. Here is what that looks like so far:

You might notice that the text on the left is not Poki Nanpa code. To begin with, I have just written an interpreter that takes colours and positions of pixels and outputs them on the canvas. From this template, I will write the actual interpreter.

The sources for this web page are in the “web interface” folder of the project repository.

As I begin to think about how the graphical output of Poki Nanpa will integrate with its written code, it’s important to consider that there will not be a console to print errors and debugging information to.

The graphical output will take the form of a grid of pixels. This grid could be too small to print long error messages to, so instead it would be good to have a concise way of representing errors graphically. In the Poki Nanpa specification I have begun outlining what this might look like. Take the following example:

Everything here is represented in binary; white or yellow means 0, any other colour means 1. The yellow pixels are just background markers to make it more readable. The red row at the top is a binary representation of the error code. Here it is 90, which in the current draft of the specification corresponds to a syntax error. The next rows represent a stack trace; the error originates in line 13 of import 2, which was called by line 42 of the current file.

The user can also throw their own custom errors if they want, just by inserting [error number] pakala into their code. pakala means “error”, conveniently.

I have also reworked the functions as discussed in my last post. For the prepositions, I’m only allowing tan, tawa, and lon. I want to reserve kepeken and sama because I think I’ll need those for other things.

Unsurprisingly, the most important part of a functional language is defining and calling functions.

The first way of calling functions in Poki Nanpa came about as a natural result of considering how numbers are written in Toki Pona. First, a word like wan should act like a parameterless function returning the value 1. But also, it should act as a post-fix function that acts on the number preceding it, such that tu wan becomes 3. I have set out this way of calling functions in the specification for Poki Nanpa, which I introduced a few days ago.

The second, more complex way of calling functions is still a work in progress, complicated by the aesthetic aim not to include any punctuation in Poki Nanpa’s syntax. In Toki Pona grammar, there are various words that can be used to separate a verb (function) from an object (parameter). The most common is the particle e, but there are also prepositions like tawa (sort of like “to”) and tan (sort of like “from”).

Here’s an example: o jo e ilo e lipu tawa soweli means “give the pencil and homework to the dog.” Notice how each object in the sentence is preceded by the e particle, except for the final one which uses the tawa preposition. This is one solution to the problem of nested functions in a language without brackets; mandate that the user must use e before each parameter and then tawa or tan before the final parameter, depending on whichever makes the most contextual sense for the function in question. “Add x to y” might use tawa, while “Subtract x from y” might use tan. Toki Pona has other prepositions too.

The other solution is to disallow functions from having more than one definition, which was my first idea and that’s what’s in the specification right now, but I feel like this is not a very fun way to have to write a functional program.

tl;dr there are three ways to call a function:

• the parameterless way: function
• the postfix way: x function
• the prefix way: function e x e y e... tawa/tan z

Technical details will be in the specification!

Since the last blog post, which was a while ago and missed a lot of important developments, the first half of the project has been submitted. I’ll try to keep it updated more often from now on. I’ve settled on Poki Nanpa as its name (Toki Pona for “Number Box”) and now begins the work of actual implementation. The first part of that, as specified in the project plan and with the self-imposed deadline of the end of February, is designing and writing a specification for the language, both as a guide for others on how to use it, and as a guide for myself on how to implement it.

To get an idea of how a programming language specification is structured I took a look at the specification for the C programming language. I own a copy that was apparently illicitly smuggled out of India at some point, according to the confidently authoritative warning label on the cover.

This specification breaks the C language up into digestible chunks, starting with an overview and tutorial of the basics, and then moving into a more detailed look at each aspect of the language; operators, control flow, functions, pointers, structures, and input/output. It concludes with an overview of the C standard library. This is a good basic structure from which to form the specification for Poki Nanpa, although hopefully mine will be a lot shorter than 272 pages.

In writing this specification, I will be able to iron out the specifics of how the language works. Amendments should be iteratively added during development as required, so I plan to host it in the same Git repository as the implementation code itself will eventually go in, located here: https://git.arts.ac.uk/24021566/poki-nanpa

The inspirations behind Poki Nanpa are laid out in more detail in the actual project submission, as far as the word limit would allow. The basic plan is to create a functional language reliant on graphical inputs and outputs using the vocabulary of Toki Pona.

I am not the first person to come up with the idea of a toki pona programming language. Part of my literature review is focused on looking at other implementations of this idea. Here’s a rundown of a few that I found on Github and the esolangs wiki:

“toki pi ilo nanpa” (lit. “language of number tool”) is a programming language that attempts to structure itself in such a way that all instructions are valid sentences in toki pona. The functionality of the language is based on Lua, and as a result it is fairly versatile, with a wide range of reserved keywords, and is not too much more verbose than a regular programming language. However, this brings into question the advantage of using toki pona in the first place, since this is effectively a transliteration of a standard programming language to use different keywords, and with additional rules added on top.

“toki pona li pona” (lit. “toki pona is good”) is a far more minimalist programming language that operates on a single queue-stack, similar to more famous esoteric programming languages like Brainf**k. This reduces the number of reserved symbols but makes it very verbose and difficult to express complex programs, which perhaps makes it similar in spirit to toki pona itself. However, it is a pain to program.

“toki pi ilo sona” (lit. “language of thinking tool”) takes a novel approach by abandoning the Latin alphabet and instead using sitelen pona, a system of hieroglyphics where each glyph represents a word in toki pona. This is an interesting approach as programs written in this language are graphically quite short, similar to other programming languages making use of non-alphabetic written languages, while still having the versatility of a traditional programming language. The main issue is that no major operating system supports the display of sitelen pona characters, and most people do not have a keyboard that can type them, meaning the language is not very portable. Keeping implementations to the Latin character set may be more practical.

Knowing and being able to criticise these approaches that have already been taken will allow me to evaluate how I want to approach the problem myself. Currently, I am leaning towards an approach that avoids the use of any symbols, but I will probably not enforce grammatical correctness to the same extent as other approaches so as not to stifle the language or make it more “ike” (complex/bad).

We’ve been refining our project ideas using the MoSCoW method; establishing their scope and feasibility by examining what they Must have, what they Should have, what they Could have, and what they Won’t have. Here’s what I’ve got so far:

MUST HAVE

• Command line interface with a runnable interpreter.
• Basic data flow.
• Basic variables.
• Basic operators.
• Simple output (e.g. text)

SHOULD HAVE

• Graphical interface.
• Graphical input and output.
• Syntax that reflects the philosophy of simplification of toki pona.
• Website for collecting/submitting programs.

COULD HAVE

• JavaScript interpreter to run programs on the website.
• Some kind of exhibitible display of some example programs.

WON’T HAVE

• Advanced programming features that can make code shorter but not simpler. (e.g. lambda functions)

I’ve also been thinking about a name for this project. “Poki Nanpa” is the current frontrunner. The literal translation of that is “Number Box”. It’s sometimes used to mean “computer” (though “ilo sona”, “knowledge tool”, is the more standard translation) and I don’t think anyone else has used that as the name of a toki pona programming language yet.

I’ve had a few ideas bouncing around in my head for my final year project. For a while now I’ve been interested in a constructed language called toki pona, and I’d be interested in taking a programmatic approach to something related to that.

What makes toki pona unique is its extremely small vocabulary of about a hundred and twenty words, each representing a wide and abstract concept. There is no mucking around with the linguistic markers that tend to increase the wordsets of other languages. There is no tense, no plurality, and no gender. There are three pronouns; first, second, and third person. The word “jan” means “person”, but it can also mean “humanity”. The word for “want” is the same as the word for “need”. This all sounds cool on paper, but in practice toki pona is a very annoying language to speak. Imagine trying to communicate anything remotely technical but you can only use the hundred most common words in the English language; it’s almost impossible. This is because toki pona is an “artlang”; a constructed language not necessarily built purely for communication. Instead, the focus of toki pona is on simplifying thoughts down to their most abstract and basic forms.

So while this is impractical for most applications, I think there is interesting potential for creating a programming language that makes use of this same philosophy of simplicity. When programming, it is easy to get bogged down in long cumbersome statements and complicated mixtures of different data flow structures. By creating a programming language that uses the vocabulary and syntax of toki pona, could you force yourself to program in a way that is simple and easy to read and write?

I’m not the first person to come up with the idea of a toki pona programming language; in fact there are already several. My preliminary literature review will involve researching what approaches these take, and figuring out how my own take could iterate and improve on their ideas. I will also research other attempts to simplify programming, and other aspects of toki pona, conlangs in general, and “esolangs” (esoteric programming languages). Embracing the “art” part of “artlang”, I may specify that this language, rather than printing to a console, will be entirely focused around drawing on a canvas. This might allow my final product to be an interactive (online?) piece, showcasing interesting outputs and perhaps allowing people to write their own programs.

I’ve begun working on a basic syntax. The title of this post is a hello world program. Here’s a more complex program, for solving the wheat and chessboard problem.

nanpa Chessboard li nanpa mute mute mute tu tu // Chessboard = 64
nanpa I li nanpa ala // I = 0
nanpa Total li nanpa ala // Total = 0
I li lili e Chessboard la // if I < Chessboard
nanpa Grains li nanpa tu // Grains = 2
o sewi sewi sewi e Grains tan I // Grains **= I
o sewi e Total tan Grains // Total += Grains
o sewi e I // I++
o awen // repeat loop
o sitelen e Total // print Total