Imagine, for a second, you're running a PW. Not the cool kind of imagining, of having a hundred players or a million areas that slithers along smoothly powered by six DMs running quests every evening. No, imagine the bits that are actual work. Renting a physical or virtual server, administrating it, managing scripting functionality, database schema, web frontends, writing nwnx code, building areas, managing custom content, and so on.

.rar yay

As it started out in the far past, the usual process of working on a PW was to send around a .mod - probably packaged up as a .rar - with people making edits and then sending it on (or back to you, for you to copy it onto the server, restart, and hope nothing breaks). You wouldn't really have backups of it (or if you did, it wasn't automated or scheduled). Edits would on occasion be overwritten by other authors, with little oversight on who made what change, who broke the latest script, or who slipped in undesired game-changing content (exploit or otherwise). You'd never do it this way, of course, but that's the Status Quo, so why not stick with it for just a moment.

Now, considering that you do software for a living, you quickly come to the conclusion that the current process sucks and you go about automating things. First the small bits - like setting up a source code repository. Then writing up some scripts to build a .mod and deploy it on the server, in varying degrees of sophistication. Maybe even convert all module data into YAML, because the diffs look much nicer this way. Of course, to do that, you'd have to write a library/toolkit because none of the existing tickle your fancy, so you set off with that and waste a few weeksmonths on it.

A Jenkins instance follows quickly because you can't be arsed to package and upload manually. Maybe Jenkins even restarts your server once no players are online (and you swiftly notice that that's a bad idea, as soon as the first DM comes a-crying about his lost quest setup). This works well enough, for a few monthsyears, but eventually ..

Imagine you have spent considerable time setting up a build chain like this, for yourself, on the assumption you will be the only one deploying content for the forseeable future. You consider yourself the gatekeeper, because - well, someone has to keep an eye on quality control, right?

Eventually, you decide that this process, in fact, sucks even more. All you managed to do is make MORE work for yourself - you spent time writing all those scripts to help you, true, but everything still passes through you. You decided to lock everyone else out of quality control, so you're kind of expected to test things yourself too! This is not acceptable.

What if I don't care anymore, with even more MediaWiki

So, you decide to re-engineer your whole process again. You have a vague vision of what it looks like - from a technical standpoint, you start eyeballing MediaWiki and wonder if you can just, like, .. adaptit to suit your purposes (figuratively speaking! Don't look at me like that!). Of course, just sloshing 500 areas into a Wiki engine doesn't work, so you start exploring options on how to go about making collaborative editing of a NWN PW a reality. You stopped caring about politics (after all, we're all adults and if the system isn't robust enough to catch trolls, why bother?) and you just focus on the technical bits. In fact, by just giving everyone write access, just maybe something fantastic will come into reality!

Your aims are laid out in a simple fashion, actually:

• A web interface that shows "some kind of map" where people can drag and drop areas to make up your world _{note to future self: make it pretty}
• Ideally, it would hook into the toolset and you could just one-click areas to upload them
• Some kind of versioned storage that takes all module data. Maybe even custom content, but that would require 2da tracking .. eh, nevermind. Let's stick to module data.
• Verification of said content, so people don't break the server by uploading invalid files, crashy areas, infinitely-looping merchants, and so on.
• Maybe some small scripts to rewrite/change content as it is uploaded.

You spend some monthstime brooding over this, while life moves on and you are generally busy/lazy, and eventually see that you actually can't do this right now in the way you want to. You'd first have to write some more glue code. You need reliable pubsub messaging (no, database polling doesn't count), and if you could hook ResMan into that to load content on the fly (via nwnx_areas which was still broken at the time), that'd be great. So you sit down one evening, in a train - seriously, what's it with trains and shoddy hotel rooms that makes you desperately productive, and start hacking on something you had no idea about. Eventually, nwnx_redis appears - a plugin linking redis, pubsub, script execution, resman and SCO/RCO together.

Various iterations of a web frontend quickly follows (well, not quickly - you decide to refactor the player-facing vault first, wasting another few weeks of off-hours work), and foolish as you are you think that you should just reinvent the wheel. A hacky handrolled database-based storage system for your module data migrates from IDEA to whereever deleted inodes go. You settle on GIT for backend storage instead, read up on rugged, and because you have a very naive, misguided vision of players writing their own tools in this shiny future of yours, you set up the frontend as a API-based angular app, with proper authentication and everything.

Even more glue code later and a million bugfixes later - memory leaks  in NWNX, your plugins, and even in nwserver itself you didn't even know existed; pointer double frees that you decided to just ignore by taping it over with libdiehard, and so on - you actually have a proof of concept working. You can upload areas (.are & .git, really) into your fancy new frontend and they magically appear in the DM chooser list. You can jump into them, see everything live; click on objects and even get data on what bugger spent time making the area. Success!

Wait, what?

Not so successful. Now you notice that this was just half the work you needed to do. Those politics you ignored at first? Turns out, it doesn't really work without them. You need some kind of logical framework on how to manage content. Those "some small scripts" you briefly thought about a year ago? They're kind of essential. You start drawing up a "module schema" - much like for a database, but for content instead. How to name things, how to organise resrefs; what can be committed and what cannot.

This project quickly balloons into a full Development Wiki, and a refactor of your whole upload/commit/deploy process (taking another few time units of off-hour evening work) so you can hook into the process at any and all points. You figure that enforcing some rules isn't so bad after all, because having a big furball made by people letting their creativity go wild is vastly different from people just not knowing better and making a hash of things.

But with all that now well underway, all it takes is the occasional bugfix or addition and things are going well enough - not smoothly, but amiably.

This is the first part in a series on FWN; just some background history on what it took to make such a simple thing. The articles to follow will go into painful detail on how the actual bits work.