Recently in Coder Category

| No Comments | No TrackBacks

I was using Lua for my final year project, but found some bugs with the Pluto library on 64-bit systems.

Pluto lets you serialise almost any Lua object into a flat representation; tables, functions, even coroutines, which allowed me to recreate continuations in Lua, albeit with some limitations.

However, the library seemingly hadn’t been updated since early 2010, and there were a few problems. Digging on the lua-users mailing list I found that someone else had taken over maintenance and posted the code on github. If I’d found this earlier it would’ve saved me a lot of time!

Hopefully future users can find it more quickly: Pluto - Heavy Duty Persistence for Lua

| No Comments | No TrackBacks

Stolen off reddit:

Look, most new products spend YEARS in development - god only spent 6 days rushing out the entire universe, and only a single day on humans (if that). When you rush things out they contain bugs and after that you need to patch them and maintain them.

Adam and Eve were corrupted by a snake (worm) and everything went south from there - you can see it time and time again; god backed up his important documents and placed them in an ark and then formatted the world, before declaring he should never have to do it again, yet he reguarly formats mini sections of the globe in a haphazard manner.

Jesus represents a sort of Service Pack, however it is clear that after that God just sort of gave up and hasn’t bothered to make any updates since.

| No Comments | No TrackBacks

To save money, I decided to migrate several of my family’s websites to a VPS. Now, setting up lighttpd is a piece of cake, but because these websites are dynamic, database-backed affairs, you have to do a little more work. I might be a programmer, but I’m not really a Linux expert.

That said, it was relatively painless, and in contrast to some previous experience. Besides, who needs tech support if you have Google and a decent brain in your head?

Why Lighty?

Apache is very well known and full-featured. It’s also rather memory-hungry. Lighttpd has a far smaller memory footprint, but still has a lot of capability; if YouTube and Wikipedia prefer it over Apache, it’ll be just fine for my piddly little sites.

The smaller memory means I can get away with a smaller VPS. I’m using an incredibly inexpensive 128MB instance from Prgmr. Even that is overkill, but then I’m not using the server solely for websites.

Also, configuring lighty is pretty simple. I never had much fun configuring Apache.

Step 0: download all you require

I chose to build lighttpd and PHP from source for more control. For MySQL, I decided to install a package provided by the IUS Community Project (more details on doing this yourself here).

Step 1: get lighttpd working

Follow the guide, obeying the instructions for your particular distribution.

It would be wise to create a unprivileged user for lighttpd to run as. After installing,

addgroup www
adduser -g www -M www

Now let’s create somewhere for the websites to reside:

mkdir /var/www
mkdir /var/www/default
echo hello world >/var/www/default/index.html
chown -R www:www /var/www/default

Finally, setup lighttpd.conf (should reside at /etc/lighttpd/lighttpd.conf) with the following:

server.username = "www"
server.groupname = "www"
server.document-root = "/var/www/default/"

mimetype.assign = (
  ".html" => "text/html", 
  ".txt" => "text/plain",
  ".jpg" => "image/jpeg",
  ".png" => "image/png" 
)

Run /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf and try connecting to your server. If you get the ‘hello world’ message, everything is rosy. Press Ctrl+C to stop lighttpd and install the other bits.

Don’t forget to /sbin/chkconfig lighttpd on so that lighty runs on server startup.

Step 2: install MySQL

I did this from a package, so nothing more to say here than yum install mysql51 mysql51-server mysql51-devel.

It’s important you install the development package as PHP requires the MySQL libraries/header files for MySQL support.

After installation, run /usr/bin/mysql_secure_installation to finish setting up the database. (Very important!) Follow the prompts and accept their suggestions.

Check you can connect to the database with mysql --user=root --password. If everything’s fine, go create a database and a user account for your PHP website to use later.

Again, /sbin/chkconfig mysqld on.

Step 3: build and install PHP

yum install flex bison
tar xvjf php-5.3.2.tar.bz2
cd php-5.3.2
./configure --with-mysql
make
su make install
cp php.ini-production /usr/local/lib/php.ini

You probably want to pass more options to ./configure depending on what libraries you need support for, or alternatively don’t want support for. A full list is here. Configure will complain if it can’t find PHP’s dependencies, but it’s not too hard to sort out (hooray for package managers!).

Edit your lighttpd.conf to add the following:

server.modules += ( "mod_fastcgi" )

fastcgi.server = ( ".php" =>
                   ( "localhost" =>
                     (
                       "socket" => "/tmp/php.socket",
                       "bin-path" => "/usr/local/bin/php-cgi",
                       "bin-environment" => (
                         "PHP_FCGI_CHILDREN" => "16",
                         "PHP_FCGI_MAX_REQUESTS" => "10000"
                       ),
                       "bin-copy-environment" => (
                         "PATH", "SHELL", "USER"
                       ),
                       "min-procs" => 1,
                       "max-procs" => 1,
                       "idle-timeout" => 20
                     )
                   )
                )

This sets up FastCGI for use with PHP. You should restart lighttpd to make the changes take effect (/etc/init.d/lighttpd restart) and test if PHP is working in the classic manner (echo <?php phpinfo() ?> >/var/www/default/phpinfo.php). Go to http:///phpinfo.php and see what happens!

Now go and configure to your heart’s content. Have fun!

Things I do

I use mod_rewrite, mod_redirect and mod_accesslog, among others. As you can see, lighty is quite powerful.

# Redirect subdomainless addresses to their www equivalents
$HTTP["host"] =~ "^([^.]+)\.(com|co\.uk|net)$"{
  url.redirect            = ( "^/(.*)" => "http://www.%0/$1" )
}

# Rewrite subdomains.domain.com to doc-root/subdomain
$HTTP["host"] =~ "^(.*)\.starfishgames\.co\.uk$" {
  server.document-root = "/var/www/starfishgames.co.uk/pages/"
  url.rewrite-once = ( "^/(.*)" => "/%1/$1" )
  accesslog.filename   = "/var/www/starfishgames.co.uk/access.log"
}
| No Comments | No TrackBacks
I'm actually making a PDF of this for my uni course, which I shall display / break up into bits when it's done. You'll have to wait a few days for that, though - be patient.

Good news, though; LuaJIT has just received all the sponsorship it had asked for to create a 64-bit version! I don't know a great deal about the differences between x86 and x86-64, but having seen some of the unique compilation and optimisation methods employed in LuaJIT, I'm rather excited to see what kind of tricks he can wring out of the 64-bit instruction set.
| No Comments | No TrackBacks
When I was a kid, I often asked "why" or "what" questions. "Why is the sky blue", "what's in this box", "why can't I have another chocolate", etc. and the answer that would infuriate me the most was "not telling!", or even worse, "because!" Yeah, okay, but WHYYYyyeeeeeeeee-aaaayyyyyyyyy? *pout*

I wonder if mathematicians feel the same way when they're faced with things like NP-complete problems.

"Yes, there is a solution to this, but hell if I'm going to tell you what it is."

"Aww, meanie! Is this a solution?"

"No."

"How about this?"

"Nope."

"Okay... um, this one?"

"No."

"What abou-"

"Nor that."

"Then WHAAAAT IIIIIS IIIIIIIIT?!" *pout*


The best you can hope for is knowing when a problem falls into that category and steering well clear of it. Or be smart enough to discover P = NP, but I won't hold my breath.
| 1 Comment | No TrackBacks
In which I learn about metamethods, try my best to restrain myself from abusing them, and get an opportunity to hurl more abuse at Java

I fancied learning a new language, and needed an excuse to do it. Fortunately, I have been assigned to create a raytracer. We'd already discussed the mathematics of it in class, and the end result is fairly obvious, so there shouldn't be too much frustration caused by boxing myself into a design corner accidentally. What better opportunity?

There were two languages hovering in my mind, Lua and Scala. Well, raytracers need performance, and LuaJIT was far more promising in this respect than Scala, so the choice was obvious. It turns out Lua is an amazingly simple language - for something that isn't Lisp or Brainfuck, at any rate.

This also means there isn't any built-in object system, should you want one. Strings are strings, functions are first-class (yay!), integers are doubles (that's much better than it sounds, by the way) and tables are the only data structure you get to play with. Tables are arrays and/or dictionaries, and while the way you access array and dictionary elements are the same - var[x] - they exist independently, for the most part, so your fears of hash-table-backed arrays are unfounded.

You also get to access dictionary elements using var.element, which is equivalent to var["element"] - a nice bit of syntactic sugar which JavaScript also has, and helps you feel as if you're dealing with well-defined objects rather than key/value stores. You can also duplicate JavaScript's prototype-oriented system very easily in Lua (more later). However, that's where the similarity ends.

Along with tables, you also get metatables. They let you customise the behaviour of a table, remapping operations upon it with your own functions. Think operator overloading in C++, except in a dynamic language. Importantly, you get to overload key accesses and writes, though only for keys that don't exist in the table inspected (for performance reasons if anything else). For example, we try to access foo["abc"], but foo doesn't contain abc. So, we call foo's metamethod for looking up table keys, which simply returns the value of the key in another table, bar. Ta-da, you've just inherited the keys of bar for free! You can do more complicated things, if you wish - but use your power with restraint.

I've seen far too many people condemn operator overloading in C++ - and often for good reason, since there are many things to consider, and many assumptions must be made such as the burden of object allocation, exceptions, and dealing with the sometimes-baroque overload resolution rules.

In Lua, however, you have garbage collection, a well-defined exception system (die horribly), and the simple rule of 'call the left object's metamethod, if it doesn't have one, call the right object's metamethod, if that doesn't have one then die horribly'. It's not complicated, and it leaves you to do something sensible, whatever that might mean in your case.

On the subject of exceptions, this isn't like Java where you are forced to catch exceptions for things which are honestly rather trivial, like a NumberFormatException for parsing a bloody integer of all things. Lua's error system is borne from its origins and primary purpose as an extension language, in that it's OK for a chunk of code to error and return control to its caller, since you probably won't have huge chunks of code anyway, and errors worth halting the whole thing are due to faulty program logic rather than edge cases. As a result, Lua and its libraries return error codes for things which aren't the programmer's fault, such as trying and failing to open a file, and raises halting errors for things that are, like accessing a non-existent member of a table. This is further encouraged because the only error control mechanisms you get are error (and assert) for raising errors, and pcall to execute a function and capture either its return value or its error.

It's certainly satisfactory enough for most uses, though I'd rather have Python's more powerful exception handling if I was writing a large program. In any case, error handling depends more on the programmer than the language. Even when it tries to force your hand like Java's checked exceptions does, it's no guarantee you actually solve the problem, let alone in a reasonable way.

There's one thing Lua does better than Python, though (and Java, but that's obvious). First-class functions with real closures. The best way I can express it is being able to pass around behaviour with state as an object like any other, and being able to define it with the same convenience. If something is easy to use, you will use it more often. Being able to define behaviour means you'll readily create functions that accept behaviours, just like map, filter, sort... and thus get one step closer to the Holy Grail of Code Reuse. I think we can all agree that the less new stuff we have to write, the easier it is for us and the fewer new bugs we will introduce. Amen to that.
| No Comments | No TrackBacks
That's it, my last deadline of 2009 has been met. It's rather too tempting to spend the day making ice cream, playing TF2 (team Demo!) and going out on the town, so that is precisely what I'll do. But only today!

Tomorrow, however... that is when I start my Coder series on

Building a raytracer in Lua

Serving both as the solution to one of my assignments and a diary of my experiences learning a new-to-me, yet very promising language, and one that receives wide usage in the games industry already. I'd do well to know it, especially since MMF2 has a Lua interpreter extension.

Raytracers are very computationally intensive, which is why they're generally written in C(++). As far as performance is concerned, LuaJIT appears to be doing reasonably well for itself, and you get the benefits of dynamic typing, closures and first-class functions. You also have a rather flexible object system - as in, there isn't really one, but the language features plus a little syntactic sugar give you the tools to develop a model that fits your particular circumstances. You know, like Perl, except without the feeling you're filling your source code with censored profanity.

A raytracer might not need all that flexibility, being one of the most naturally-object-oriented applications you can make, but I certainly don't intend to make raytracers all my life. Or ever again.

Plus, since I'm not Steve Yegge, the lack of braces isn't a turn-off. Besides, the benchmarks I've seen put the V8 JavaScript engine slower than LuaJIT - though I have no idea whether that holds once you make a more substantial program.

At the very least, compared to Win32 C programming, Lua will be quite the welcome break. Once you've had a taste of functional programming, writing in C again seems nearly as painful as if you'd lost your compiler and had to resort to typing COPY CON: OHGODWHY.EXE (and believe me, typing italics into a command prompt is tricky stuff). Time will tell whether it's all I've cracked it up to be.
| 1 Comment | No TrackBacks
Reading an article on Chrome OS prompted me to ponder the up-and-coming juggernaut that is Google - at least where the Internet is concerned, but as that keeps growing, so will their power.

The obvious thing is that Google is about advertising. What do you think pays for all your space on Gmail? The revenue from Google Apps for Business won't cover it - and if you think of all the ISPs that resell the platform yet retain ads, e.g. Sky, you can see where the real value lies.

The added benefit of reselling is passing on the cost of support; this leaves more resources for Google to spend making those ads more relevant and thereby improving the bottom line.

That's fine, though. I can live with that. Companies need to make money somehow, and Google doesn't do this in a particularly insidious way; their ads are the only ones I actually allow because I have found them useful at times, and because they aren't insanely irritating compared to the competition. (Even YouTube fares better to commercial TV, IMHO.)

The real concern in my eyes is that, while Google is sticking to 'Do no evil', they're not doing good, either. Let's get back to Chrome OS: no support for native apps or data storage is in no way good for users. It is a serious step back in time in an era where storage is insanely cheap and mobile processors have more power than huge mainframes did not too long ago. People might spend huge amounts of time on the web, but everybody I know needs at least one killer app which you can't get via a web browser. That might be Skype, it might be an image editing program... hell, people have mobile devices which need to sync somehow, whether it's an iPod, smartphone, or digital camera.

I just wonder where the value comes in from having Google's glorified browser as the only thing on your netbook. It's not like manufacturers haven't tried putting Linux on to cut costs - look at how successful that ended up. I suspect people want a little more than that - but I also suspect I may be proved wrong and Google succeed over that original failure.

But that's not really the issue here. There's a good reason why Google want people to stick to the web - it's what they're good at, they make money from it, and it's all they really care about. The linked article at the top gives plenty of examples where it's obvious their only loyalty is to themselves. Getting people on the web where they have huge popularity and more control of things can only be good; even if some people avoid all Google-using sites as a consequence, there's going to be many more using them by virtue of their reputation, and they know this. A win for Google.

Advertisers are their customers, and users are their product. They worked out that attracting users means you have to make things users like, and damn anyone else; the perfect example being Google Book Search's lack of regard for copyrighted works, and the associated furore.

Sure, Google makes nice things for us. Just don't fool yourself into believing it means they love you; if you do, one day you'll wake up with a nasty surprise, a very awkward situation on your hands and terabytes of data all about you stored somewhere in the Googolplex. The sea level will rise a metre solely from how much advertisers are salivating right now.
| No Comments | No TrackBacks
Things are taking a turn for the weird. My lecturer for Advanced Software Development has informed us how he believes design patterns are a marker of the deficiencies of a language. (With which I agree wholeheartedly.) He is now proceeding to teach us design patterns.

I'm not sure whether this is a case of 'know your enemy' or Clockwork Orange-style indoctrination to get us to foam at the mouths if somebody so much as mentions a facade, but either way, I think I approve...maybe?

Maybe the powers that be have forced him to teach Java against his wishes, and he's doomed to spend the rest of his days teaching us how to write a unit test with a mock Factory Visitor watching over his shoulders, the Chain of Responsibility weighing heavily on his shoulders. If I were him I'd implement a Proxy of myself and insert it into lectures...
| No Comments | No TrackBacks
CAVEATS: This is 110% opinion with heaps of experience thrown in.
As an aside, in the real world there is far less conscious design than you realise. Sometimes this is a very good thing because it reduces confidence - mantras are like car sat-navs, rely on them totally and one day you might find yourself driving off a cliff because you trust it perfectly. YMMV (especially if you leave the road and plummet towards the ocean).


"The new keyword isn't inherently evil, but it may complicate your design later." Emphasis on may. (Context: using new Connection(db) over Connection.get(db) )

For example, say you have a database connection object and you want to set up a pool of X objects that you want to reuse. Rather than use the factory method, you could edit the Connection object such that is composed of e.g. a DatabaseSocket which interfaces with the database directly. Connection's constructor grabs a DatabaseSocket from the pool, and existing methods on the Connection object forward themselves to the DatabaseSocket. (delegation!)

From outside it's essentially the same, the methods do what you want them to, no knowledge of the DatabaseSocket required... the only difference is each Connection object can hold additional state as well as a reference to the (possibly shared) DatabaseSocket.

This could be very desirable, especially in multi-threaded scenarios. Consider the case where there is contention for a limited resource. With a factory method that directly hands you an object, the onus is usually on you to properly handle sharing and locking for that object. (It's not as simple as adding 'synchronised' to all the methods!) In addition, if other objects in the pool become unused but the object you were given is highly contended, you can't switch to another object unless your code specifically accounts for this. And, if your code accounts for this, the entire factory pattern is less of an abstraction and more of a distraction.

However, if you construct an object unique to you, one that delegates your commands to a shared resource, that object can alter its behaviour with respect to the resource, even storing command history and seamlessly replacing it with another resource if need be (consider distributed systems and a failing link, or simply a process that has crashed), you maintain the abstraction that you have *a* instance of the resource, which is often all you want.

Sharing objects is nice, but instantiating a brand-new object is always more powerful because in the worst case it can be a perfect proxy for another object, and thus replace the factory pattern entirely.  Personally I see the factory pattern as an obtuse version of early optimisation, and as we all know it is a fool's game. We have refactoring for this kind of thing anyway, and at the very least, you can always s/new Connection/Connection.get/

My rule of thumb generally (and completely ignoring most uses of the factory pattern) is: 'do you want a brand new object of your very own, or are you happy being given an object to achieve what you want?' If you're doing anything with mutable state outside your program logic, you probably don't need to track any extra data outside of whatever exists externally, and you should ask whatever subsystem you are dealing with for appropriate objects to this effect. However, if you want additional data with your ice-cream sundae, the former is actually a sensible thing to do, and doesn't preclude you from changing the implementation of the created class so that it grabs the 'real' object elsewhere behind the scenes.

Besides which, factory methods won't magically make everything work if you change the underlying implementation. Integration testing still needs to be done, you almost certainly need to fix bugs in the edge cases.  Anyone who has done multi-threaded programming will attest to the nightmare caused when you start changing things like this.

-----

Symmetric operations like Equals are not solved satisfactory by pure-OO approaches. Verbs are verbs, they exist without nouns if need be - Java makes this hard, but if you can, treat verbs like first-class beings.  c.f. CLOS multiple dispatch. (Not that this is a perfect solution either, but I scribed this for future reference.)

Composition is often a good thing to use, but that really depends on what you're trying to achieve.

Liskov substitution principle - if you can adhere to that you can safely derive your class.  Otherwise you will almost certainly run into errors later down the line, some quite gnarly and subtle - imagine if you program all Animals to jump(), then derive an Elephant from Animal...

Inheritance has its uses, but you need to be careful about what it really means. It's not about extending functionality as much as it is about telling people "Hey, I have a cool object here with special features I can use, but since it behaves exactly like this more basic object, you guys get to play with the same instance as me!".

Aside: as far as defensive programming is concerned, you can learn a lot by programming to a closed-source API like Windows. By the third time they screw you over by making some subtle change, you learn not to trust implementation details too much. (And besides which, classes change, code does, libraries... that's what integration testing is for, surely?)


Imagine you buy some skittles from the shop one day. Next week, you go in, and you try and do the same thing, but the shop keeper punches you in the nose before telling you "oh, if you want to purchase your item, you have to say 'Blooblebloo' and twirl the golden baton three times". No, you'd call the police for him being a menace to society and your now-bloody nose. Do you honestly expect your software to be any different?

If you change your interface such that invariants, preconditions and postconditions change, strengthen or weaken respectively, you will negatively effect other code up if they rely on it. This is not a fault with inheritance, interfaces, OOP or whatever - you told the programmer you'd always do something one way, deal with the consequences of changing it to something completely different.  This is a seriously bad design flaw.

Aside: The Old New Thing is a blog about backwards-compatibility in Windows among other subjects, and reading that will prove many times over that taking interfaces at more than face value causes enormous problems. To be fair to the zillions of win32 developers, they had no power to change the APIs to expose hidden features they needed, so this kind of trickery was sadly inevitable for many years. You, on the other hand, have no excuse in open source-land.


Incidentally, Eiffel needs a big mention because it hasn't done in the lectures yet. Look up 'design by contract'. It's like first-class unit testing. It also eases a lot of the above issues by tightly coupling guarantees about your code with the code itself - you can't break them without it knowing about it, and likewise, calling code knows exactly what to expect from you.

Anyway, the trick is to design your interfaces so you give code as little guarantee as is possible while allowing useful things to be done. This gives you more freedom to change the implementation, add new methods, etc. assuming the calling code is bug-free and makes no wrong assumptions about your interface.

Think about real-life contracts which lock down so much and try to close loopholes. There's a good reason why they do that, annoying as it is, so perhaps you should consider closing the ambiguity in your own software so that everybody is playing to the same rules.

What annoys me about interfaces though is that you're forced to do a fair amount of donkey work to use them. Think of the high-water-mark stack example and my suggestion to override all the push() methods so that it works. Doable, not pleasant, though, is it?


A superior solution to interfaces - and to the occasional twisted desire to have multiple inheritance - is traits, as implemented by Squeak Smalltalk and Scala on the JVM. Traits promote behaviour reuse rather than the combined state+behaviour reuse of inheritance. Mixins are similar, slightly less powerful and harder to manipulate in complicated cases, but at least are available in languages like Python and Lisp. Scala lies between these two with its own traits implementation (and, as a JVM language, has the benefit of being compilable, which I am sure will please Ian immensely and hopefully encourage him to look at it if he hasn't already done so!)
More on traits: http://scg.unibe.ch/archive/papers/Scha03aTraits.pdf
Hopefully someone will find that enlightening.



And most importantly, there is more to program design than OO!