The object of this project is to create an open, free (as in libre), robust framework for the rapid production of 2D games. This includes a generalized game editor (distributable to users for modding), and any other external bits required for the game development/distribution process.
The engine must fit the following criteria:
0. Preferrably, the engine should utilize middle layer APIs (or inherited class methods) to seperate itself from any specific API. This ensures quick porting to alternate APIS.
1. The core code of the engine must be public domain or under a BSD/MIT-like license, or be cheaply licensable for modification without redistribution (for compatibility with modern copy protection products).
2. The engine should not be designed as dependent on a core library that does not satisfy criterion 1, unless that library is seperated from the engine code by an intermediate API layer- i.e. LGPL libraries are fine if they are called from an API layer that is not the engine code nor the external API. It may be noted that such a layer would make it easier to port the engine to an alternate API.
3. The engine must support scripting.
4. The engine must be capable of reading data in a myriad of formats OR have a "free" (as in libre) toolchain to convert to native format from most standard formats.
5. The engine must support compressed assets, decompressed either at initialization or in a streaming fashing during gameplay/level load, or both. A corollary, the engine must support encryption of such data in a similar way.
6. The engine must be written in portable C or C++, without any endianess or 64-bit support problems. At a minimum, Windows support, Mac OS/X support, and linux/bsd support (the latter in at least a minimum way).
7. The engine should support low bitrate audio and video compression codecs. The toolchain to utilize this capability must be inexpensive or free.
8. It would be preferred that the engine be data driven, i.e. simply acting on data provided at startup for the majority of the content rendered.
9. Preferrably, the engine is lightweight enough to become a browser plugin (for direct access to content distributed/streamed via the web/http/bittorrent).
10. Preferrably, the engine is capable or can be modified support the ability create gameplay content within itself; i.e. one can interactively adjust the gameplay during playtesting. This goes hand in hand with the ability to create levels within the engine. This has an important bearing on rapidly producing high quality gameplay content.
Is there an engine that meets the criteria above?
Sections 3(e), 3(f), 3(g) make it clear that a game produced with Torque 2D would tie the hands of it's developers to allow user modification, or derivative works (in the case of a free title released under the Creative Commons license or similar).
This relies on Direct3D, violating the portable requirement.
This is windows only. While satisfying several of the criteria that the other engines do not, this is the least extensible of all the engines I've examined.
Again, DirectX dependent, and thus windows only.
RPG Maker
A lack of an official community has prevented me from looking into this further- but it is limited and proprietary, although satisfying many of the criteria that game maker does.
Tile Studio is not an engine per se, but a sprite/tile editor with tilemapping capabilities, that runs on win32. It outputs an open file format that the gameplay engine would then load. This is not a very portable format (few other pieces of software support the output that Tile Studio generates), and violates the principle of in-engine editting.
There are many other bits of software out there claiming to be "engines", but they are glorified helper libraries that do precious little more than the standard choices in libraries (SDL, Allegro), and are often far less portable.
But wait! What about modifying a 3D engine to display 2D things?
1. It is usually a bad thing when you twist an environment to do things it wasn't designed to do.
2. I do not wish to require even limited 3D hardware support. Many of the platforms I wish to support have segments with such hardware (slightly old/cheap laptops, many Macintoshes, integrated video chipsets, PDAs). The need for this requirement is rapidly fading in the PC space, however.
3. This pretty much blows any hope of the engine being lightweight enough to serve as a browser plugin.
It becomes clear that if I am to satisfy my criteria, I must create this flexible/extensible solution myself.
As this is a big job, I must use 3rd party APIs wherever I can, to reduce the total lines of code to be written, reducing the number of bugs, and speeding up the development process. I will aim for a feature complete version 1.0 of the engine to run on most PC OSes (Win32, Linux/BSD, Mac OS/x).
1. SDL seems to be a good choice. It, and helper libraries like sdl_gfx, sdl_console, sdl_net, and others support the greatest portion of functionality I need. While the breadth of capability is greatest, the fewest high level features (sprite collisions) is directly supported by SDL or a helper libraries. SDL also has weird issues with threading, unless the SDL_Fastthreads (linux only at the moment, has race conditions) is applied.
2. Allegro is the next strongest contender as a core choice; as it is an older library, it seems to have some legacy issues API-wise. Its community is smaller than SDL's; as far as I can tell, it has been used in far fewer "production" (i.e. commercial) environments than SDL has. Allegro is under a "giftware" as-is license, and so can be taken into proprietary environments where the LGPL licenses of ClanLib and SDL cannot go.
3. Clanlib is the third strongest, with no OS/X support as of July 2005. It however handles more of the high level issues (sprite collisions, better integrated networking library) than SDL or Allegro.
Clanlib and SDL are both distributed under the LGPL; Allegro is distributed under a compatible license. Thus, if I made a derivative work (a new SDL helper library for instance) by combining inlined code to save time, the code would be usable legally if relicensed under the LGPL (my intention anyway).
As I have the most experience with SDL, I will use it initially under my engine's API Abstraction Layer. Due to the issues with threading, I will have to be careful to design this layer to truly shield the core from the API, or issues could crop up later if threads need to be supported. To this end, I must plan the alternatives.
Alternatives to SDL and it's helper libraries:
1. OpenAL supports simple software 3D positional sound, as well as mixing many channels (SDL, SDL_mixer). It takes wavs, and these are generated by whatever library we are using to decode the format our sound is stored in (fmod, libvorbis). Alternatively FMOD is now capable of far more than decoding to waveform, and could be a general cross-platform replacement, or an upgrade in engine capability at a later date (3d sound for example). FMOD has also been ported to the current crop of consoles as of July 2005.
2. HawkThreads/Hawknet/Hawkvoice: robust open source libraries that would replace SDL_net and provide OS-independent posix-style threads (shields the programmer from windows threads).
3. Allegro and clanlib could be leveraged for hardware-independent display of strictly 2D graphics; otherwise os-specific libraries will probably be employed. SDL replacement is a likelyhood on some limited devices anyway (PDAs, cell phones, consoles), so Directdraw on windows, some simple X windows code for BSD/Linux, and ?!? on Macs.
Of course, when you want multiple threads, its highly likely that your hardware is going to support OpenGL nicely; alternatively, you are on a high-powered game console using the native libraries and the SDK anyway.
Outside the realm of what SDL covers, is metadata, I/O, choice of scripting language, video codecs, sound codecs, image formats, and text rendering.
Metadata is a no-brainer for making it easy to import/export data from the engine into other tools, and for keeping the organization of game assets largely outside of engine code (rather than passing the lower-level non-engine API (SDL) the filename, have the data loaded at startup or on demand and pass it once its loaded). XML is a no-brainer, and the license and lightweight of TinyXML means we can take take it anywhere.
OS/device independent I/O is difficult. In the PC space, stdio gets a little funky when you combine it with the likes of file permissions and differing directory structures. To that end, the standard directory layout places the engine executable, config file, and it's shared libraries/.dlls (if any) in the base directory, with game assets in a packed (possibly encrypted) format in a "base" directory a la Quake 1-3(4?). Supporting this simple standard on various OSes should be easy, and we can have a "virtual" file layout inside of the .zip file. This does of course require a middle layer API to shield the engine from the outside implimentation.
1. Zlib is the obvious choice for this, and has a very permissive license. It has also been used commercially in game development for awhile.
2. 7z is an alternative format, and supports higher ratio compression via LZMA. It however uses a less permissive license (LGPL), and is a unknown quantity compared to the likes of zlib.
The decision of scripting language seems most difficult.
1. Python seemed obvious... but its the heaviest weight and is a whole language on its own. If I developed many in-engine tools with it, I would be beholden to this beast.
2. Lua is very popular in the game development community. It is relatively lightweight, has good external tool support, and is easy to extend.
3. TCL has a lightweight interpereter that compiles at runtime, the language is extendable by callback as are most scripting languages, and several "shell" environments have been written for it- a boon for an integrated console in the engine.
4. There are a myriad of other predeveloped solutions, but the unfamiliarty with these on the part of the community at large and myself make these unviable in the long run.
5. Roll your own. A rule/event/object property-driven language would be perfect for quickly developing games and reusing script code between them. This is not a good fit for automated tool generation by flex/bison/etc. however (or at least I never got it to work). This however was my last project, and I worked on it distractedly for 14 months without getting anywhere. I feel that this is a dead end.
Like these other peripheral libraries, the scripting engine will be (relatively) easy to create a plugin architecture for. I think I will go with TCL as I am most familiar with it.
Video Codec
Video Codecs were a sticky point for free/low cost software development for some time, until the Theora codec was pioneered by the Xiph.org foundation as part of Ogg project. It performs comperably to mpeg4/divx style formats, and is under a BSD license without patent encumberance.
Sound Codec
With the advent of mp3, high fidelty audio at small size and reasonable CPU consumption became reality. Vorbis, developed by the Xiph.org foundation as part of the Ogg project, is a no-brainer for audio where high fidelity is required, such as music or speech. It's license is of course the same as that of Theora, and it has been used time and again in commercial production environments.
Image Format
As Wikipedia has some excellent pages covering this subject; I will simply link those pages rather than listing all of the (known to me) pros and cons, so here are the important bits.
1. JPEG: This is encumbered by many patents and commercial interests. It is also very lossy, and this makes it undesirable for high-resolution sprites, tiles, and backrounds. It's successor format, JPEG 2 or JPEG2000 gets rid of the quality problems at the cost of yet more patents and higher CPU cost.
2. GIF: Still slightly patent encumbered, a standard gif only supports 8-bit (can be palletized) color depth. Up to 256 frames of animation standard, slightly handy for storing animated sprites.
3. PNG: Non-patent encumbered, this open format supports alpha channel and lossless compression (deflate). Well supported, and in 1996 became a W3C recommendation for prefered graphics format on the web.
4. TIFF: TIFF is more of a container format than an image format per se; this means that many pieces of software do not correctly load many TIFF files, as there are many possible variations. There is a lot of overhead, but this is a very flexible format overall. Unencumbered by patents.
5. Targa: Targa is behind all the rest of the files listed in terms of usage, but it's relatively simple format means it is supported just as well. It supports 1-24+alpha bpp images, a built-in color key for transparency, and several other amenities. It is also unencumbered by patents of any form.
Targa seems to be the best overall choice. While it offers no compression other than RLL, it is presumed that the files will be packed together in a single archive, so this is no disadvantage (and is an advantage vs. PNG or some variants of TIFF in terms of load times and code complexity).
Text rendering. This is important for menus within the engine, gui interfaces for complex games and integrated tools.
1. SDL_ttf and other libraries that take a string, a ttf font file, and a few arguements to create a bitmap of the string of text. The biggest hurdle here is coping with layout, as there is no *easy* way to know before rendering the number of pixels wide and tall a random string of 30 characters will be in any given font/point size. This library depends on Freetype, which is not exactly lightweight.
2. Bitmapped fonts. These can be easily created from truetype and other fonts in external tools, and you have the advantage of knowing the exact size of a displayed string.
I see way more bitmapped fonts than truetype/other scaling formats in commercial games and it is far simpler to impliment... bitmapped fonts then.
This concludes the Engine Manifesto. This document was meant to spell out my requirements, convince existing engines do not fulfill them, and to spell out the fundamental technologies needed to realize a 2D engine meet those requirements. This is not an architecture document.
The Architecture Manifesto (much shorter) will be posted publicly here at some time in the near future. If it interests the reader, this software is minimally feature complete, running only on Linux (with a mingw port lagging a week or two behind), and can do anything that a SNES could do, at much higher resolutions (the standard is 1024x768) and fidelity, with networking. Currently, work is progressing on the engine's internal editor (all scripted with TCL), stamping out a few bugs, and on collecting a toolchain to produce all content.
There will soon be a Toolchain Manifesto. It turns out that a few bits of software are only free/open/inexpensive on *ix. This includes software to grab/edit video, some capability in sound editting (Audacity does frustrating close to enough), good debugging, and should the user elect this, software to transcode proprietary formats into the open ones the engine understands. I realize the places *ix occupies in the hearts of many users, and for both users that love and hate, love to hate, or love to love Linux, I am preparing a Knoppix-based bootable CD distrubution with all the programs and utilities required to develop software with the Engine (including the engine).