Senzee 5

Senzee 5 Stolen!

2011-09-20T10:09:00.003-04:00

My domain, senzee5.com, was stolen!

Now it points to some Japanese SEO company, a bunch of jerks. It had expired, I guess, end of July and it wasn't set to auto-renew and it looks like I missed the emails and they snatched it up.

ARGHHHH!!!

I'm at paulsenzee.com for now..

Cholera

2011-05-24T20:15:00.024-04:00

They talked of the days of the cholera in Santiago Miahuatlán, a village fifteen minutes from the city of Tehuacán de las Granadas in the southern edge of the state of Puebla close to the border of Oaxaca. In those days there were wives and husbands, brothers and sisters, small children, mothers and fathers just gone. The water was unclean and there is no deliverance from water. In all they said one-in-four perished in this town, but I did not believe it. It could not be that in the 1990s a medieval epidemic could wipe out a town like this with not a peep heard elsewhere.

It may be true. The gobierno underreports cholera outbreaks. The tragedy was unbearable to this place, whether one in four or one in forty, because compounding its numbers is only a multiplying of infinity.

Miahuatlán's torment could be seen in the still dirt streets, the ramshackle homes at the periphery, the dust-blown open market and in the Catholic church of broken walls that stood as the once-elegant headpiece of the town square called the zócalo. Among the agave plants that bowed to the dusty earth, cacti stood tall - like the people here themselves - even encumbered with the weight of overripe red pitaya fruit, round and wet with a taste and texture almost like watermelon.

Three years after the epidemic, sweat soaked our shirts as we raced again in the sun past men with straw hats plowing and harrowing new milpas and breaking up dark soil beneath the pale dust. We slowed to saunter at a reverent pace as we walked those streets near the graveyards, where from time to time other men with straw hats, muscled from years of labor, would stand quietly with calloused hands and infected nails clutching their deep-lined faces or kneel at wooden crosses and weep.

Ambition

2011-05-04T11:01:00.001-04:00

The Relic (iTunes) was released April 14, 2011 to disastrous reception. It sold well at first but sales fell off after the crushing reviews. The most common complaint was the control scheme but there were many others as well. For now, I have an update ready for the controller and some collision issues. I will update Crystal as soon as I get hold of the new build and submit the update then. I'm also working to update other aspects in the meantime.

It is a disappointment. Many people wanted to play The Relic. The game was ambitious from the start - especially given the resources I had to develop it. However, to make it in the App Store, you have to have an ambitious game and I knew that. You also have to have a polished game, and The Relic - as it is right now - is not that.

In any event I'm committed to The Relic and little by little, with each update, it will fully become the incredible game that's hidden a bit below the surface. I recently read this article in gamesindustry.biz (which may require registration/login) about indie project budgets. The author maintains that to make a successful indie mobile game you ideally need $100k in resources. The Relic was developed in my bedroom after work hours. There were long periods that I didn't make progress on it at all because at the time I was working insane hours at my day job. To date, I've spent less than $3,000 on it. I haven't yet recouped that investment but we'll see what the future holds.

The Relic holds a great deal of promise and although it was made and will continue to be made for obscenely little money and resources, we will see it grow into something tremendous.

The Relic

2011-03-16T17:02:00.024-04:00

In the App Store April 14, 2011!

The Relic is a 3D Gauntlet- or Diablo-style arcade RPG developed by Axolotl Studios (aka me) and published by Chillingo. It runs on iPhone and iPad. It's taken some time to develop but it is currently in submission with Apple and will be released within the next few weeks. I'm proud of it. There are those projects that you create that turn out far better that you had hoped and this is one of them - for the most part. Like all your creations that you know intimately, you know its flaws. One big disappointment was having to cut multiplayer.

Multiplayer is not in this release. I spent months working on it and got it working reasonably well but the architecture was a mess and it was hard to fit the rest of the game around it at that time and it was getting later and later. I was starting to believe myself that the game would never be released, so we cut it. I intend to get it in in a future update.

And Death Shall Have No Dominion

The two things with the most impact in this world are love and death. Attachment and detachment. We all die. Yet in this era, there exists a mere possibility that even death itself may yield to the relentless advance of technology. The Relic explores the concept of immortality and of mastery of the biological in an unexpected, although pretty minimalistic way. I want to add more depth and nuance to the story in future updates.

The Art of The Relic

Mark Jones created the character art and most of the animations. Some of the environment art (the tents and market area) was stock 3D art and well worth the little I paid for it. In the future I will use stock art more. I created the dungeons themselves and props such as the crates and barrels and the items that can be dropped, some of the animations, the UI art and the logos. Much of the scene art, the intro, etc. uses art adapted from public domain art sources - that is, famous, old works of art. I used Blender for the 3D art.

The Music of The Relic

Most of the music in The Relic is stock music, however, The Relic includes two songs by indie songwriter and musician Andy Livingston, "Indigo Winter" and "Heavy."

The Code of The Relic

One of the most successful parts of The Relic's engine is the sheer number of animated 3D enemies that it can handle. The Relic engine uses Lua heavily for game-side scripting, but it is organized in a way that does not interfere with The Relic's ability to manage so many enemies.

I'll write more about coding concerns later, but with 438 source files (not including the Lua files), this is certainly the largest software project I've written entirely by myself. It's written primarily in C++ and I did most of the development on Windows PC with Visual Studio and occasionally testing with the iPhone with platform-specific pieces written in C++ on the Windows side and Objective-C on the iPhone/iPad side. As with Indigo Ocho, near the final stages of development, development shifted completely over to the Mac/iPhone.

The Relic (iPhone 4 screens) [on facebook]

Axolotl Studios

2011-03-16T16:44:00.012-04:00

Here's a quick update. It has been a long, long time.

I left EA and I'm now working at XOS Digital making mobile apps for all sorts of platforms which is a blast and reasonable hours too. My latest side project has been my indie iPhone/iPad/iPod dev shop, Axolotl Studios. For my day job I'm now working at XOS Digital making mobile apps for all sorts of platforms which is a blast and reasonable hours too. I'm currently fighting a lot with BlackBerry.

A couple of years ago now (wow) we/I released Indigo Ocho, a 3D accelerometer-based labyrinthine puzzle game, which is not currently in the App Store, but I do intend to get it back there again. Now finally, I have completed The Relic to be released shortly by Chillingo. I'll cover these games separately in subsequent posts.

Indigo Ocho

2011-03-11T23:54:00.014-05:00

Indigo Ocho has been out of the App Store for some time now, and now that The Relic is done, I'll try to get Indigo Ocho back up there.

(Indigo Ocho: more coming soon)

Indigo Ocho [facebook]

Echo

2011-03-10T20:57:00.000-05:00

elvira_jones has initiated a conversation with joebob10_.
23:50:00 elvira_jones: how did ur surgry go last wk?
23:50:23 joebob10_: ok they got the tumor out of my head. it was small benine. no pain tho hpy to be hm
23:52:01 elvira_jones: :) good i was worried bout u
23:58:13 elvira_jones: good nite im checkin out
23:58:42 joebob10_: u2 thx :)
The conversation between joebob10_ and elvira_jones has ended.
joebob10 has initiated a conversation with joebob10_.
23:59:59 joebob10: who r u?
00:00:02 joebob10_: who r u?
00:05:48 joebob10: hello?
00:05:51 joebob10_: hello?
00:07:59 joebob10: ([gibberish], 'seriously who r u', 'WTF?')
00:08:02 joebob10_: a;lsdfklj;kasdfkl;j
00:08:02 joebob10: WTF?
00:08:05 joebob10_: WTF?
00:08:05 joebob10: ([gibberish], 'elvira?', 'dude u creep me out', [profanity], 'stop it!')
00:08:08 joebob10_: stop it!
00:08:08 joebob10: im serious
00:08:11 joebob10_: im serious
00:08:12 joebob10: ([gibberish], 'is this a joke?')
00:08:15 joebob10_: alskdfj;asdquoioieruw3482
The conversation between joebob10_ and joebob10 has ended.
joebob10_ has initiated a conversation with docPhillips_.
00:15:12 joebob10_: hey dr p? sorry to bother you so late i didn't know if ur awake
00:15:15 docPhillips_: How are you feeling?
00:18:27 joebob10_: i think im having delusions weird ims from a guy named with my old username
00:18:30 docPhillips_: Oh, yeah.. Hey don't worry about that - it's part of the tests we told you about.
00:19:12 joebob10_: no no this is different he types what i'm going to type in the future before i type it!
00:19:15 docPhillips_: Ok, so the thing is.. Do you remember the form you signed saying that we could use any data we were able to gather from your surgery for?
00:19:49 joebob10_: yeah sure
00:19:52 docPhillips_: Well, we modeled your brain in a computer. I guess Dr. Minsky left it on all night. I'm sorry.
00:23:00 joebob10_: no way
00:23:03 docPhillips_: Yeah, we'll turn it off first thing tomorrow morning.
00:27:52 joebob10_: no u dont understand no this isnt right this creeps me out
00:27:55 docPhillips_: Really, I promise, we'll turn it off in the morning. It won't bother you again.
00:30:04 joebob10_: no ur using my brain???!!! thats not rite at all dude im gonna sue ur ass
00:30:07 docPhillips_: Don't be upset. Look, we'll talk about this in the morning, get it all straightened out.
00:31:13 joebob10_: no im goin to call my lawyer in the morning to get all over this shit
00:31:16 docPhillips_: Julia,
00:31:18 joebob10_: dude wrong chat window
00:31:19 docPhillips_: could you or one of the other nurses turn off joebob10_ please? Thanks.
00:31:20 joebob10_: ?

Industry Broadcast: Podcasts of Senzee 5 Articles

2008-11-29T13:16:00.004-05:00

A Senzee 5 podcast (Audio Article #34: Concurrency in Game Design + Why Use C++?) is now available at Industry Broadcast (http://www.industrybroadcast.com/) read aloud by Ryan Wiancko. There's a lot of really great stuff there, so get over there and check it out!

The audio article posted combines my entries Concurrency in Game Development and Why Do We Use C++?

Industry Broadcast is the result of Ryan's great idea to offer insight from game developers in audio format. He says -

I was toiling over one disappointing aspect of almost every one of my days: That being that I have so little time outside of my 16-20 hour work day to be a human being that I never am able to sit down and read enough of the articles being published online about the games industry. Occasionally my RSS feeder throws something at me that I just have to make time for but for the most part I see the headlines of features and articles and have to regretfully click away and focus on the 65 new emails sitting in my mail box. I thought ‘God, if I could just have an LCD in the shower with me I might actually have the ability to digest the plethora of useful information regularly being shared by the top minds in our industry.” And then an even better idea struck me: If all of this material was available in Audio Format then I, as well as fellow developers, could be listening to these amazing articles whilst working out or doing the dishes, or or or.

Ryan also maintains a personal blog here.

Underengineering Undead

2008-11-16T00:16:00.097-05:00

We learn more from our failures than from our successes. In A Developer's Life Stages, the first stage is The Underengineer, in part inspired by this poignant failure.

As a college freshman in late 1992, I started making an Ultima VI-like role-playing game.

In researching this post, I went through my journals from that time period to try to figure out exactly how Undead failed. One glaring issue is that ~75% of the journal was about girls, with the remainder divided among writing music, painting classes and working on the game. While totally expected for a distractible teen, given the difficulty of the task and a rapidly approaching deadline (discussed later) it's not a good indicator of eventual success of the project. Nevertheless, software development is a human activity and must coexist with other human activities. Initially, I was more interested in producing an Ultima-esque demo that would get me a job at Origin Systems.

I met a classmate in my speech class at school named Alex Kapadia. Alex had some experience with sound programming and was a game fanatic. I showed him what I'd built, he was thrilled and we began collaborating. At the time, we were playing a game called Solar Winds distributed by a small shareware company called Epic MegaGames before it shortened its name and became known as the world-class developer of Unreal and Gears of War. Each time the game ended, up popped an ad by Epic requesting developers to send in their games for publishing. We sent Undead to Epic and awaited a response. We didn't have to wait long. In early summer 1993 we knew that we would be working with Epic. We sent several versions to Epic over the following year. As it started to gel a bit, we began working more closely with them.

JOURNAL 12/06/93: Talked to Tim Sweeney (president of Epic MegaGames) today. Wow, he knows games. He gave me a lot of good suggestions. He's looking into an artist and musician for the game. He was talking some intense MONEY!!!!!!!!!!

Tim estimated that the game would make Alex and I $40,000, which was an unfathomable amount of money for me as a 19-year old.

JOURNAL 12/08/93: Today Dan Froelich, musician and composer for Epic called me. He lives in Kansas City (that's crazy!) and works for Informix. He told me a lot about how the game publishing thing works. After finals week, we'll get together.

We did end up getting together, Dan, Alex and I at Manny's, the finest Mexican restaurant in Kansas City. He told us a lot about Epic and the business of developing shareware games. He also brought beta copies of Epic's Xargon and Ken's Labyrinth, developed by wunderkind Ken Silverman.

JOURNAL 12/15/93:

From Tim Sweeney (via E-mail):
Date: Wed, 15 Dec 1993 04:28:24 -0500 (EST)
From: Tim Sweeney <70451.633@compuserve.com>
Subject: RE: Undead & Pinball
To: PSENZEE@VAX1.UMKC.EDU
Message-id: <931215092823_70451.633_fhg40-1@compuserve.com>
Content-transfer-encoding: 7BIT
--
Thanks! It's great that you and Dan live in the same town. I hope you'll be interested in working with Dan - he's been with us on Jill, Kiloblaster, Brix, Solar Winds, Adventure Math, and Xargon and his experience working with these projects would be helpful in addition to his music! I'm looking forward to seeing more of Undead! It's the best "first" game any author has shown us. Judging by your coding and artwork, your game has great potential and you have tremendous potential as an author. Undead still has a long way to go in developing a story, creating all the artwork, and turning it into a fun, unique, and successful shareware game, but it looks like you have the perserverence to see it through to the end. Keep up the great work!

-Tim, Epic MegaGames
(3:22 AM - normal business hours for we game developers) :-)

A House of Cards

In late 1993, we believed Undead was going to be an enormous hit. The possibilities appeared endless. Even so, I was starting to deeply fear that I wouldn't be able to finish it. Undead had grown in complexity so much that modifying it was becoming difficult. It was approaching 20,000 lines of C and assembly and because it had no higher level architecture to speak of, I was losing my ability to understand it. Disparate pieces were tightly coupled. There was no layering of subsystems. Significant numerical values were hard-coded everywhere. Every line of code I added made it harder to work with. It was like going from troweling wet concrete to chiseling hardened concrete. This was a classic case of underengineering and I was unequipped to fix it. I was scared. Instead of confronting that issue, I began obsessing over the art - most of the development time was now spent in DeluxePaint getting the art just right.

JOURNAL 03/02/94: Undead is gaining bugs as I try to fix it! Argh. And it's having memory problems -- specifically, not enough static memory.

Worse yet, I had a deadline that I hadn't yet shared with Epic. In a year, more or less, I'd be leaving the country for two years without access to a computer. Now, I suspect I could have fixed Undead's woes given enough time, but I didn't have that time.

Man on a Mission

I grew up in the Mormon (LDS) church. Although I left the church many years ago now, at the time I was devout and determined (and expected) to become a missionary (I'll write sometime later about why this was the right choice, despite my eventual leaving). Around March 1994, I called Tim and told him that I had committed to leave for a two-year mission to central Mexico sometime around September. Understandably, he wasn't happy. I'd naively assumed (wishful thinking) that I would be able to get everything done before leaving in the fall. He knew better.

JOURNAL 03/26/94: BAD NEWS. Tim Sweeney got the copy of Undead. He told me he thought it was fantastic, but that there was no way on earth I was going to get it done before August. In addition, he said it would be absolutely obsolete when I got back. 03/27/94: I didn't realize how upset I was about what Tim said about Undead not being done until I woke this morning and I had been having nightmares about it all night. Of course he's right and I knew it anyway.

Obsolescence was always on our tail and we felt its pressure even during development. The style of game that Undead represented was an Ultima VI class game when the RPG state of the art was Ultima VII and in 1993, Doom (as shareware no less!) splayed the writing on the wall for all 2D games.

Perhaps, as a project, this was doomed from the outset for many reasons. The most obvious was my leaving before it was done. Compared to what I build today, though, the game was not at all complex. It should have been straightforward to build it in a year-and-a-half. It's been said that programming ability manifests at a young age, while skill in software architecture comes much later. In the end, I knew the dark secret. It wasn't merely unfinished, it was unfinishable.

Geek History #1

2008-11-15T16:35:00.054-05:00

Circa 1987-1991

Connecting with old friends on Facebook lately led me to remember those early days as a teenager writing video games. It's incredible how much has changed since then. It was nearly impossible to get any information on how to program a computer, much less on how to write games. The library books were typically user and application-oriented. C and assembly books could only be had at the bookstore at ~$60 a piece. In the days before bookstores came with sitting/reading furniture, being a kid without a lot of cash, I'd go to Waldenbooks in the mall and stand there reading, absorbing as much as possible before I had to leave. Today, on the other hand, there's rarely a need to buy technical books because whatever I need can be found on the web faster and for free. Today is definitely better.

With a tiny bit of programming in school (Applesoft and Commodore BASIC) when we got our first computer (a Tandy 1000SX), I naturally started with GW-BASIC. It was too slow for the kinds of games I wanted to make - games like Ultima V and Times of Lore.

For Christmas, I asked my parents for a BASIC Compiler (a what?, they asked). It was what I'd circled for them in a magazine - Turbo Basic. I expected to drop my games in and have them run like crazy. There was a hitch, however. The Tandy 1000SX shipped with an OEM version of Microsoft's GW-BASIC that natively supported the Tandy's enhanced CGA video adapter (essentially the same as the the IBM PCjr's with the 16-color 320x200 mode). Turbo Basic didn't. So, as long as the games could be confined to the horrible four-color plain CGA, they'd compile just fine and run pretty fast. I wrote a paint program (Paint!, image below) and a couple of plain (and simple) CGA games (Alien Invasion and BreakOut!, below) in Turbo Basic. In the end, Turbo Basic was still not fast enough and four colors weren't sufficient.

More importantly, there was one specific technical issue that simply could not be solved with any sort of BASIC. I wanted a four-way scrolling world, like those in nearly every commercial game of the time. I was so desperate to make this sort of game that I spent what little money I had on a commercial library (for BASIC) that supported scrolling through assembly-language routines. It turned out that the scrolling supported by the library was simple text scrolling through the BIOS routines, wholly inadequate for what I wanted. I was extremely disappointed and called the (apparently) single-person company for a refund. He refused. There was only possible solution - to learn C.

C had (and still has) a reputation of being extremely difficult, yet it was the language most commercial games were written in. I bought Mix C - a command line compiler/linker and nothing else - for $20. Coming from BASIC, it was a difficult environment. After floundering with that for months and maybe a year, I finally broke down and bought the Turbo C compiler for $150 and spent the next two weeks trying to build a working piece of software in C. Although I didn't fully understand pointers at the end of the two weeks, I wasn't far off from a basic working knowledge of C. Good tools can make all the difference. I still couldn't figure out scrolling until I received a letter from Herman Miller that brought everything together. From that Legacy I (shown above and below) in C finally came to be. It was a collaboration between Eric Lambert (who did the maps and much of the game design) and I. In truth, it wasn't a game in the end - the player is merely able to roam the world in full 16-color Tandy splendor. That was good enough for me because we now had a new computer with 256 (!) colors. It was time to start all over.

Ufo Hill

2008-11-09T22:40:00.010-05:00

Atlixco's colony of Metepec is surrounded by conical hills, strange low emerald mountains, that rise steeply from the plains in a way that seems man-made. Arrieta and I took our free day to climb the close hill that towered just meters from our house. For their uniform shape, I often imagined that these hills were ancient buried Aztec structures like the pyramid at Cholula covered up by Cortes after his domination of Mexico.

The locals here and apparently the UFO community worldwide came to regard these hills and this area as sacred earth for extraterrestrial sightings. UFOs, called OVNIs in Spanish, had been spotted in these hills and around the cusp of the temperamental volcano Popocatepetl just beyond many times. The real explanation was certainly something far more mundane - such as helicopters or fighter jets.

To reach the hill we skirted a waterway, climbed a sloppily mortared rust and gray brick aqueduct and stepped through, plant by plant, the overgrown garden of the village backwoods. The hills weren't nearly the rich green up close that they were from a distance. After a couple of hours of wading through the dry yellow brush we reached the top. We searched for the UFO landing spot in reverence but found nothing at the top of the sacred hill but brush.

Metepec is astounding from that vantage point, and we could see much more of it than we had ever visited. Arrieta and I sat at the top for some time, saying nothing. I don't know what we were hoping for.

Eventually, Arrieta said, "A soda?"
"Yeah."

We climbed down, bought two Peñafiel Manzanas from a local vendor and chatted with him a few minutes. We returned him the bottles and went home. It was a great day.

El Cerro de los Ovnis

La colonia Metepec de Atlixco está rodeado por colinas cónicas, extraño montañas esmeraldas pequeños que se elevan abruptamente desde la llanura de una forma que parece hecha por el hombre. Arrieta y yo tomamos nuestro tiempo libre para subir a la colina cercana que se alzaba a pocos metros de nuestra casa. Por sus formas uniformes, a menudo me imaginaba que estas colinas fueron antiguas estructuras aztecas sepultadas como la pirámide de Cholula cubierto por Cortés después de su dominación de México.

La gente que vivía aquí y al parecer la comunidad OVNI en todo el mundo llegaron a considerar estas colinas como tierra sagrada para los avistamientos extraterrestres. Ovnis habían sido vistos en estes cerros y cerca de la cúspide del volcán temperamental Popocatépetl y más allá muchas veces. La explicación verdadera es sin duda algo mucho más mundano - tales como helicópteros o aviones de combate.

Para llegar al cerro que bordeaba un curso de agua, nos subimos un acueducto de ladrillo de color rojizo y gris y pisamos, planta por planta, el jardín invadido de la selva virgen del pueblo. Los cerros no tenían el verde intenso de cerca que estaban a la distancia. Después de un par de horas de caminar por el pincel seco y amarillo llegamos a la cima. Con reverencia buscamos el lugar de aterrizaje de ovnis pero no encontramos nada en la parte superior de la colina sagrada sino pincel.

Metepec es asombrosa desde ese punto de vista, y pudimos ver mucho más de lo que habíamos visitado nunca. Arrieta y yo nos sentamos en la parte superior durante algún tiempo, sin decir nada. No sé lo que estábamos esperando.

Finalmente, Arrieta dijo: "¿Un refresco?"
"Sí."

Bajamos, compramos dos Peñafiel Manzanas de un proveedor y conversamos con él unos minutos. Le devolvimos las botellas y nos fuimos a casa. Fue un gran día.

La Casa de los Ancianos

2008-11-08T20:38:00.002-05:00

Arturo and his sixteen year-old daughter drove Arrieta and me to help at a house of the elderly in Atlixco. We rode in his ancient blue Chevy truck inscribed with El Pollo on the door, a name which had come to refer to Arturo himself. We flew past palms that line the old road to Axocopan and past the lush hills and rich lime colored milpas - fields of corn - with rickety seats rattling. Arrieta sat on a seat of wood fashioned to replace the broken seat springs. Juliana, El Pollo's daughter, sat in the truck bed holding down the supplies.

I sat in the middle of the cab knocking the stick with my knee. El Pollo pointed the way to the far side of Atlixco below the high hill where an aged limestone Christ the Redeemer watches over like Rio de Janeiro's famous statue. We came to a home for older, abandoned seniors, wedged within a contiguous block of lemon and mango hued buildings.

An iron gate sealed an opening a little wider than the breadth of a car, and we approached it. A priest opened the gate when we knocked. He knew our companion and was visibly pleased to see him.

"You're here again, are you Arturo?" the priest asked. A short, stocky nun in her habit came out of their living quarters and joined the conversation.
"It's been some time since you've both come hasn't it?" the nun began, referring to El Pollo and Juliana.
"Sí, sí...we have been busy."
"This place doesn't stay clean by itself .." she trailed off, realizing that the cleaning of the home was in the end her responsibility and no one else's.
"Sí, sí...we know," said El Pollo as he disappeared to fix the home's plumbing.

El Pollo was a good man, he'd founded the Alcoholics Anonymous group in Metepec, our village outside of Atlixco, and was tireless in his service to his community although he was not a man without error. I admired him. His work in this home for the elderly had been one of his projects, and it was difficult. We'd forged an unexpected alliance with him: we were religious, he was not. El Pollo was a Mexican Jean Valjean, and he had taken it upon himself and his daughter to care for the dying señores.

We entered the home's courtyard of the colonial style, gray cobblestone groundwork with a large stone well of water in the center and a faded rainbow of drying clothes stretched across between opposite roofs. In each room around the courtyard, with no doors or curtains, the boarders moved slowly in the shadows. In the first of the rooms I reeled from the stench - feces, decay and the pungent ammonia odor of urea.

Since arriving in Mexico and passing countless days of food poisoning and intestinal infection, I had a weak stomach. I convulsed and heaved spontaneously.

"Careful, amigo! Get some air," Arrieta said and pushed me back out the door.
"Yeah, I'll be alright, thanks," I responded.

I regained my composure and breathed shallowly so as not to catch the room's odors. I looked around. Two mattresses were on the floor; one was on a bed frame. Each mattress was alive with the sheer number of flies. Urine, feces and blood were smeared all over the elderly women's beds. One was sitting up, and two were lying.

These three women lived in varying states of incomprehension and did not immediately acknowledge our presence. The sitting woman accosted me after a moment.

"Hijo - son,...why did you leave me here?” she pleaded.
"¿Cómo?" I asked, bewildered.
"Why did you leave me here to die, hijo, do you have no love for your mother?"

She wore an old, dirty green and blue plaid dress with an apron and nodded to herself, muttering continually about her son or her husband things we couldn't understand. I had never seen someone so sad. Even so, she was the most capable of the three and was fully dressed.

The señora in the bed parallel to that of the sitting woman - Pilar - was dressed with a t-shirt, a shawl and nothing more. There was a bunched up blanket soiled with weeks of fecal matter that she pulled slowly over herself when we she finally saw us.

We stepped around the puddles of urine on the floor and we watched with admiration as Juliana took charge and began cleaning rooms, changing and feeding them. She gently directed us to take the señoras, one by one, undress them, shower them, and then redress them. We started with Pilar. She spoke with fits of nearly unintelligible Spanish.

"What are you doing to me?" she demanded.
"We're going to bathe you, señora" Arrieta said.
"No, no, no, no, no, no..."
"It'll be alright, don't you worry, you'll be fine," I said. We began undressing her.
"No no no nonono..."

She was humiliated and I was ashamed. Pilar had lost her family, the integrity and clarity of her mind, certainly any shred of dignity she had ever possessed. We walked her emaciated body naked, smeared with excrement and encircled by flies, across the courtyard to the shower in view of everyone in the home. I helped her while Arrieta prepared the next señora.

There was no other option. On our way to the shower as we inched our way over the uneven stones, lifting Pilar's feet up lightly at some points, higher at others, something beautiful and surprising happened. Leaning upon the stone and concrete framework of the gray, massive well, there was a small old bent-over man with a cane, wearing no shirt and only tattered, frayed briefs.

He spoke to Pilar in a way I recognized instantly and as they exchanged awkward, tender words something wholly unexpected appeared in the home's cloud of dementia.

"How are you, my lovely Pilar?" he asked her, tipping an invisible hat.
"I'm fine! And how are you?" she responded, momentarily radiant.
"I feel good too!" he said, quivering subtly.

This old bent man fancied Pilar and she him. They spoke like shy children but as if both were fully clothed and in charge of their mental faculties. I doubt each noticed that the other was nude or nearly so or any detail of the circumstance they were both in. Between a naked old woman held up only by my right arm, perhaps in her mid-eighties, who moments before had been lying in her own waste, and this doubled and nearly unattired old man came something inexplicable.

In their shared and mutually understood lives, in the most unexpected of places, had sprung what appeared to be love.

We finished bathing her and her room companions in the shower. The abandoned mother - whose son I was - cried and muttered "no..no..no.." for the duration of the shower. The last was silent and docile. Either she did not understand what was happening, or had resigned herself to this routine performed by new strangers each time and had come to accept it.

Juliana cleaned their room and mattresses and changed their blankets, then came to assist us in the shower. With effort we finished and quietly we rode back home to Metepec.

Gangrene

2008-11-05T15:53:00.031-05:00

We ate with an older man in Amalucan and his wife many afternoons. His wife had taught me to properly wash clothes with a washboard which saved me many hours and I felt grateful. One day the man showed us his toe. "What do you think?" he asked. It was gangrenous. I was twenty and dumb but I knew that. We came by more often and each time she asked us to give him a blessing with the laying on of hands for healing. We had the priesthood, the authority to act in the name of God. The times I came with Fuentes*, he insisted on praying and always promised a complete restoration of his foot. I elbowed him as he spoke since this was impossible and he should not promise it. "In God all is possible," he told me afterwards, annoyed.

The elderly man was diabetic. I could see that he would lose not only his toe, but his foot as well - with dry black and pus-filled tissue that reached well up into his arch. Our new district leader, noble and faithful and from North Carolina came and gave him a blessing and we were hopeful. If any blessing could heal him and stop the progression of the disease it would be the one from him, because he was very good.

On a subsequent visit, we learned that he had been to the hospital and his foot had been amputated. Later, we returned to find that he was missing his lower leg. When I finally left the area his upper leg had been removed.

It disheartened me that the blessings did not halt the disease but his wife continued to ask us even after her daughter rolled her eyes and said, "no better than the witch doctors," and she said the same of the medical doctors. The grave is inevitable I guess.

I spoke to him this last time with sadness and hiding tears and he spoke with a resignation that infuriated me. I thought of my father's funeral not long before and my sister recited his favorite poem, "Do not go gentle into that good night." She cried through it and we only understood: "Rage, rage against the dying of the light." He had wanted to urge his own father at the end, but my grandfather died suddenly too with no chance to fight or say goodbye.

I tried to translate the poem for the old man but I could not capture its beauty or the dual sense of its title so I said nevermind and gave up the way we all do. Young men with all the world in their eyes don't understand why anyone would give up or waste this life for the next but old men with their lives behind them, their bodies failing and sometimes eroded by shame, regret and disappointment do understand.

Months later, I came back to the area to visit and he had died some time before. The wife was living with her daughter and took comfort in her faith.

(*name changed)

Gangrena

Comíamos con un hombre mayor y su esposa muchas tardes en Amalucan. Su esposa me había enseñado a lavarse bien la ropa con una tabla de lavar, la que me ahorró de muchas horas y me sentí agradecido. Un día nos mostró su dedo del pie. "¿Qué te parece?" nos preguntó. Era gangrenosa. Yo tenía veinte años y era tonto, pero yo lo sabía. Cada vez que llegamos nos pidió que le diera una bendición con la imposición de manos para la curación. Tuvimos el sacerdocio, la autoridad para actuar en nombre de Dios. Las veces que llegé con Fuentes*, él insistió en la oración y siempre prometió una restauración completa de su pie. Le dí un codazo mientras hablaba ya que era imposible y que no debe prometirle eso. "En Dios todo es posible", me dijo con molestia después.

El anciano era diabético. Pude ver que él perdería no sólo su dedo del pie, pero su pie entero también - con el tejido seco, negro y lleno de pus, que llegó así hasta en su arco. Nuestro líder de distrito nuevo, noble y fiel y de Carolina del Norte nos acompañó y le dió una bendición y teníamos la esperanza. Si alguna bendición pudiera curar y detener la progresión de la enfermedad sería la de él, porque él era muy bueno.

En una visita posterior, nos enteramos de que había estado en el hospital y su pie había sido amputado. Más tarde, volvimos a encontrar que le faltaba su pierna. Cuando finalmente salí del barrio había sido su muslo eliminado.

Me decepcioné de que las bendiciones no detuvieron la enfermedad, pero su esposa continuó pedirnos aún después de que su hija puso los ojos y dijo, "ni es mejor que los brujos", y ella dijo lo mismo de los médicos. La tumba es inevitable, supongo.

Hablé con él esta última vez con tristeza y con lágrimas ocultas y él hablaba con una resignación que me enfureció. Pensé en el funeral de mi padre antes de no mucho tiempo y mi hermana recitó su poema favorito de él, "No entres dócilmente en esa buena noche." Ella lloraba mientras que hablaba y sólo se entendía: "Rabia, rabia contra la agonía de la luz." Él hubiera querido exhortar a su propio padre, al final, pero mi abuelo también murió muy pronto sin oportunidad de pelear o decir adiós.

Traté de traducir el poema al español para el hombre viejo, pero yo no podía captar su belleza o el doble sentido de su título en inglés así que le dije no importa y me dí por vencido en la forma en que todos lo hacemos. Los jóvenes con el mundo en sus ojos no entienden por qué alguien querría renunciar o perder esta vida para la próxima pero los viejos con su vida detrás de ellos, y a veces erosionados por la vergüenza, el remordimiento y la decepción, lo entienden.

Meses después, volví al barrio para visitar y descubrí que el anciano había muerto algún tiempo antes. La esposa vivía con su hija y tuvo consuelo en su fe.

(*nombre cambiado)

Stages of An Entertainment Medium

2008-11-03T08:57:00.009-05:00

Some thoughts about the lifecycle stages that entertainment media seem to have in common. Let me know your feedback and (please!) counterexamples.

Novelty

The medium is brand new and typically reliant on new, immature and disruptive technologies. The cost of content creation is almost nil and, accordingly, poor quality. Content creation is the domain of individuals rather than companies. The new medium itself is revolutionary enough to sell what little content exists. Content is expensive to the consumers and margins for content creators can be large. Whatever stylistic conventions exist for content here are borrowed from other media (as film borrowed from stage, the novel borrowed from the romance). (However, because of early technical limitations, video games now look more like movies than they did in their infancy.) The conventions of the new medium begin to establish themselves and innovation is high, although craft is low.

Specialty

Content quality increases somewhat and innovation is on the rise. No mechanisms are in place for quality evaluation besides word-of-mouth. Technology is still immature and further developments in technology change content. The cost of content creation is still low and corporations are displacing auteurs as the primary producers/creators of content. Much more content is sold and price remains high. Margins are large.

Vanguard

Highly influential quality content appears. The underlying technology matures and content form innovation wanes replaced by quality craftsmanship. Evaluation of content becomes important to consumers because of an explosion of choice. Critics, a concept of literacy and a vocabulary of criticism appear. The medium becomes mainstream and its influential content becomes a part of pop culture. References to the new medium become common in older media. Margins are enormous. An 'arms race' of craftsmanship appears and as a result, dominant content creators (corporations) competitively establish enormous barriers to entry for new participants.

Maturation & Refinement

Vast quantities of content flood the market. Content drops tremendously in price. Evaluation is absolutely crucial to weed out all but the best. Poor quality content is effectively ignored by consumers. The time it takes to consume the content is more valuable to the consumer than the price paid for the content. Margins decline as the cost of content creation spirals upward. Because of enormous cost and risk, innovation is confined to craft and polish. The medium is no longer 'new' and is vulnerable to disruptive technologies.

Eventually, there is a vast landscape of low-to-zero cost content. Only the best is ever consumed. Cost of creation is enormous and margins are slim. Related services, merchandising, advertising, product placement, endorsement employed to increase profits. Core innovation is low, craft is very high. The technology is mature and is vulnerable to disruptive technologies. More and more is spent against diminishing (though differentiating) improvements in quality.

Convergence

The medium becomes a component of a more expressive and newer medium, while continuing to exist on its own.

Recorded audio becomes a part of movies

Radio (audio) - a component of television

Movies and recorded audio - a component of video games

The Consultant's Law

2008-09-13T11:16:00.003-04:00

I spent about half of my career working for Kansas City's Metro Information Services, then Keane (when they acquired MIS) as a consultant. As a consultant, I eventually learned a crucial truth which applies to consultants and employees alike. It's just that employees have the luxury of forgetting this from time to time.

The Consultant's Law

As directly as possible, ensure that your efforts make money for your clients.

That's the secret to really long contracts and an high level of control over your career. There's a well known evil doppelganger of this law that I'll call The Law of Job Security.

The Law of Job Security

As directly as possible, ensure that your absence will cost your clients money.

A lot of people seem to live by this law. It's subtly different from The Consultant's Law, but The Law of Job Security breeds resentment and comes across as unprofessional and opportunistic. It's also dangerous, because as irrational as it is in some cases, people tend to punish those who take advantage of them - even when it costs them and their company.

Poker Revisited + ESPN

2008-09-08T16:33:00.017-04:00

I hadn't given much thought to poker hand evaluators lately when I received a comment today on the post 7 pointing me to this absolutely fantastic roundup of open hand evaluators at Coding the Wheel.

James Devlin clearly put an enormous amount of work into this analysis of currently available evaluators. As part of that work, he finished up my 7 card evaluator I posted here a year-and-a-half ago! He offers it as a library with a number of other evaluators including the phenomenal 2+2 evaluator.

And of course, I also really appreciate James' comments :) -

Paul Senzee is well known, in poker-programming circles, for his perfect hash optimization to the original Cactus Kev evaluator and for being an all-around programming BAMF.

Honestly, after that great post and the 2+2 evaluator, I'm not sure if there's any more I could possibly add to the problem of poker hand evaluation.

Although In Other Projects..

Something I've done a little work on recently - At ESPN, Play-by-Play Goes Virtual.

OpenCV + Self-Organizing Maps

2008-08-05T10:51:00.007-04:00

A little on self-organizing maps and the machine vision library OpenCV..

Self-Organizing Maps

Excellent tutorials on Self-Organizing Map theory -
The Self-Organized Gene (Part 1)
The Self-Organized Gene (Part 2)

OpenCV

Intel's Open Computer Vision Library
Recently took a vacation and spent some time in Key West, which was awesome. When we got home I did a little yard work and played around with OpenCV, trivially writing a program to automatically rotate photos by detecting faces and their orientation. Now I'm hooked.

If NetFlix Rents Games?

2008-08-05T09:13:00.006-04:00

I really love NetFlix. So much that I've considered ditching many of the DVDs I own. And since starting NetFlix, I have bought very few.

NetFlix renting games has been rumored since its inception. Other services, such as GameFly have stepped in to try to become the NetFlix of games. But while NetFlix is mainstream, I suspect that GameFly's subscriber base is far from it - and it is an additional charge and an additional service if you already have NetFlix. Unlike movie makers, game makers don't have an equivalent to a 'box office' take. So if NetFlix were to ever decide to rent games, while initially great for the consumer, I believe it would profoundly impact game developer's revenue. Which would, in the end, be bad for the consumer given the skyrocketing cost of game development.

So how do we, as game developers, develop a NetFlix-proof business model? Is it all MMOG's from here on out?

Pair #1

2008-07-30T13:20:00.013-04:00

Finally, I'm putting up what I have for Pair, my script language intended for homebrew game projects. Pair is designed to be a small, embedded, acceptably fast (probably not yet the case) script language that will ultimately allow multiple concurrent VM (and not OS-based) threadlets, generators and continuations.

I don't have a lot of time on my hands, so stuff like this takes quite a long while. Now, there's some messed up stuff going on here. You'll have to forgive me that or wait until it's refined more. For example, scoping isn't right at all. The (let ..) expression isn't how it's supposed to be in a lisp-type language. It defines a variable for the rest of the outer scope which isn't right. I'll fix that. Continuations aren't currently supported, but the underlying machinery is mostly there as well as the mechanism for running multiple lightweight VM threads although this is not accessible from the command line program. Which is probably good, since I'm not sure how well that works with the garbage collector at this point. Also, there's a lot of just general messiness going on. There are a bunch of warnings at compile time and comments are sparse. These things will be fixed eventually. I've compiled these with Visual C++ .NET 2003. There are two parts here.

Version 0.01.00

The Pair Compiler
The Pair VM

See the Code Use Policy if you are interested in using any of this for your own purposes.

Code Use Policy

2008-07-30T13:16:00.008-04:00

If you use code from this blog for non-commercial use please credit Paul D. Senzee and include a link to senzee5.com. If you make changes, please send them back. If you'd like to use code from this blog for commercial purposes, contact me so that we can work something out.

Edge of Ability

2008-07-18T00:37:00.009-04:00

Some of the most successful projects are at the bleeding edge of their implementor's ability. Absolutely as simple as possible out of sheer terror of complexity.

The sequel to this phenomenon is the universal second system effect.

Intellectual discovery can be a powerful motivator in design decisions. Take care when tasking a developer with a mundane or boring task. People have a way of making uninteresting problems interesting!

I have more to say about this, including a crazy idea about the SR71 Blackbird, Lisp and the Apollo Missions all somehow appearing before their time (like from the future ;) )..

Infinite Flexibility

2008-07-16T02:25:00.023-04:00

Think about that and weep.

The Ghost Box

2008-07-09T00:39:00.087-04:00

The Art of Pranking Part II

The bigger, badder wireless pranking device that evolved from Part I of our pranking bonanza. This is a somewhat technical post. A following post will detail our exploits with it.

The completed device

Learning from the Past

There were a few issues with the first model.

The photocell was clearly visible on the front of the adapter and that made it look suspicious.

High maintenance. The contraption didn't work well when it was dark outside. In the interest of simplicity, it was designed to sound when it got too dark. That is, in the normal case, when a shadow fell across it. How dark was too dark was adjustible with a screwdriver. Of course, that meant that Ryan had to turn it off each night or if it rained out. This also made it high maintenance, because it didn't turn off automatically - it would keep sounding as long as the target was casting a shadow over the photocell. This drove Ryan crazy, since he had to listen to it and turn it on and off all the time.

It was not wireless. Fazeel was eventually able to follow the cord back to the noise box when he figured it out. Then the jig was up.

It only emitted one single type of noise. Although deliciously annoying, it didn't offer as much flexibility as we'd like.

Solutions

The photocell should be better concealed. Bought a Nortel adapter at Skycraft Surplus for $5 apiece. Using a cordless saw, opened it up, removed the internals. Cut the transformer from the prongs that go into the wall socket, leaving enough plastic around them to keep them solid, and taped the terminals up to insulate them thoroughly from each other and the rest of the electronics I was stuffing in the box. Took one of the detectors from the toy (see below), pulled out the electronics and put them in.

Cut off the antenna and hooked up the adapter's cord as the new antenna. Added a battery holder and two AAA batteries. Drilled out a hole for the sensor, slipped it through. Still pretty conspicuous.

To obscure the photocell, cut clear contact paper into a rounded-rectangle shape consistent with the types of stickers you might find on these things. Then, colored over the contact paper with black dry erase marker.

Because this adapter now has batteries (the adapter from the previous prank didn't since it was powered from the box it was attached to), I had to install some sort of switch to keep the batteries alive. Otherwise, I'd have to tear that thing back open all the time to replace them. I really didn't have a way to solidly mount a tiny switch, so I slipped a tiny reed (magnetic) switch in the seam. It's powered on by sticking a really tiny rare-earth magnet on the switch.

The device should roughly detect motion, specifically a rapid change in light intensity as opposed to merely detecting dark. This was nailed with the solution to the wireless problem described next.

Wireless would be ideal. This appeared to be beyond my skills and/or experience in electronics, but I found a toy at Toys 'R' Us that, appropriately modified, was wireless and sensed motion. It's called a SpyGear Wireless Tracker. This also had some issues when used in this way. For example, when triggered, it had a blinking red light (LED) that continued to blink until manually reset. That made my job with the glue circuitry a lot more complex.

Plays back a recorded message. This appeared to be the easiest to accomplish. Radio Shack, in fact, sells a voice recording/playback module for about $10 (above). This device was almost perfect. The difficult thing about it was that it had a rubber plunger switch to playback with no leads or any apparent way to hook into and trigger it from other circuitry. So I had to take the switch apart, drill tiny holes in the PCB for the wires and solder them directly onto the traces underneath. Not ideal. Sound quality is not great, but then I didn't really expect that much.

Finishing Up

The receiver mess (with voice record/playback unit) in nearly completed form.

Getting the glue circuitry working correctly was a pain, in the end I went for an inefficient but dead simple approach that made all the pieces work together. Maybe in the future I'll refine it. Anyhow, for now, I'm pretty happy with it.

More to come on the fun we've had.. !

What the @#$*! is that Noise?

2008-07-08T13:59:00.017-04:00

The Art of Pranking at EA

Video. Fine moments in pranking at EA Tiburon. You'll see narration by my former manager, Rob Hyder, who pulled off perhaps the finest prank ever here - The Disappearing Door.

Our prank may not be as cool as some of those, but we're still refining it. The target? Fazeel Gareeboo, my manager.

1. Misdirection

Fazeel was out for two weeks, so he was likely expecting some sort of prank. So we had to do something obvious. There's a certain report that Fazeel hates, so of course we papered his cube with it.

2. The Idea

The real idea was to set up an annoying noise that went off whenever the target scooted his chair in to work. Fortunately, Fazeel sits next to the window. So I made a detector that set off a sound whenever a shadow is cast on it. Camouflaged the detector by hollowing out an electrical adapter and used the wire from the adapter to connect with the control circuitry and the buzzer.

Then we taped the buzzer box (yes, made from a Wal-Mart soapdish) to the back of Ryan Burkett's monitor. The box sported a switch so that Ryan could flip it on and off at will.

We got three solid days of torture out of that baby! Yeah!

So, of course, I built a bigger, badder, wireless version with voice playback that I'll post about later.

Quetzalcoatl's Return

2008-07-07T23:59:00.002-04:00

As the sun set in the orange haze west of the Aztec pyramid, a bright evening star appeared overhead. The farmers - the campesinos - ran for their families and gathered them together in the fields. In an hour, the star tripled in brilliance and an enormous crowd gathered from the farms to the Zócalo. News helicopters began to circle the radiant point above them. This is the event that a score of cultures now struggle to understand.

Last night was the arrival. The campesinos whispered "It's Quetzalcoatl, returning home!". Even now, no one really knows what - or who - arrived. The descención plays all over the news this morning, and last night on live television. In the city of Cholula somewhere in central Mexico a radiant golden-robed figure descended to earth from the heavens over a period of about three hours into Cortes' Cathedral of New Spain atop the city's great half-buried pyramid and then disappeared.

Each detail was captured by camera crews from Puebla and Mexico City. The figure was of human form, fair-skinned and wore a hooded robe and a blonde beard. Every gesture was recorded by local film crews' standard-definition cameras and has been looping continuously on television for the last twelve hours. In the US, news stations digitized and cleaned the original analog broadcasts and consequently produced gigabytes of data describing this event adapted for HD. It's the essence of our age - we know exactly what happened, but not what it means.

CNN is running through scores of experts and religious leaders to answer these questions. Representatives from at least a dozen world religions (including all of the Abrahamic faiths) weigh in on the meaning of the event. Most dismiss it as an elaborate hoax. Yet for each worldview, there are those that assign it profound meaning. Here in the Southern US, that meaning is most often imminent alien invasion followed closely by some sort of precursor to the rapture.
"Is this a hoax? Is this an extra-terrestrial?"
The consensus: A hoax? Absolutely. An alien? Definitely not! Appearing human?!

Why now?

A startled televangelist blurts, "God no, what if it's Him..?"

"Is this sort of thing possible with state-of-the-art technology?", the anchor asks a panel of well-known technologists and receives mixed replies. Asking whether or not this is an act of (or is) a threatening god, one technologist answers, "I don't think we'd ever call him God or even a god. If he could kill us all, or some of us, we'd do what he says, but we'd call him a terrorist, not a god. After all, we ourselves are capable of killing off the human race..."

On the television, images of churches filled to standing-room-only cycle. In one such shot an Alabama preacher and community leader responds to the technologist's earlier comments, "Down there somewhere in old Mexico, He has come. We build flying machines, we go to space, we make thinking machines the size of my little toenail and I've even had highly educated medical doctors tell me that we can raise the dead. Are we saying that, 'No, He can't come again, He didn't in fact, because He is incapable of miracles astounding enough to make us believe He has returned?'"

Over one thousand individuals and groups across the world claim ownership for the stunt. Of them, Niels Rasmussen, a Danish engineer and little-known illusionist maintains a website detailing the technical description of how it was performed. He writes, "Success! An insane amount of traffic here. My advertisers will be happy! ;)"

* * *

Now, long after the rest of the world has forgotten, as the sun sets each evening in the haze west of the pyramid, the campesinos till their fields and wait.

Illusion: Tilbagelevering af Quetzalcoatl

2008-07-07T23:33:00.030-04:00

Illusion: Tilbagelevering af Quetzalcoatl
Niels Rasmussen, Tryllekunstner
Hillerød, Danmark
1 maj 2005

This page is in Danish. Would you like to translate it?

Illusion: The Return of Quetzalcoatl
Niels Rasmussen, Magician
Hillerød, Denmark
May 1, 2005

An illusion to convince the world of the return of Quetzalcoatl, the Aztec god. To be performed in Cholula, Puebla, Mexico or possibly Teotihuacán, near Mexico City.

Designed to be captured prior to the transition to HD with standard-definition NTSC cameras and televised to further obscure any visible wires or machinery, enhancing the illusion. It may be a couple of years before we are prepared to execute.

Levitation

Executed like David Copperfield's flying illusion, but suspended from two 'news' (Puebla) helicopters. Helicopters' motion will appear a little erratic, as if flown by inexperienced pilots, but their motion is carefully choreographed and they are flown by computer, allowing the pilots to jump in if anything goes wrong. Actor is suspended from many fine wires from each of the two helicopters. The computer controlled winches always 1.) compensate for the erratic motion of the helicopter and 2.) keep the wires in tension, in correct orientation, properly aligned and ensure that the actor is always steady and descending at the proper rate. A third news helicopter flies directly over the actor to show that no wires suspend him from above.

Radiance and Lighting

High-intensity ground-based lighting, computer-tracked from the top of the cathedral, and areas around the zocalo. Consists of reasonably high power red, green, blue lasers scanning over the actor's translucent diffuser robe. Should produce brilliant whitish light.

This plumed serpent won't have feathers or scales.

June 2008 - What's Up?

2008-06-16T09:48:00.007-04:00

Haven't blogged for a while. We've finished a bunch of games and we're now in our off-cycle here at EA Tiburon. In May or so finished up writing a C++ test for ReviewNet, which is more time-consuming than it seems.

Created a new blog for stories and entries that don't really belong here. It's called Illustrated Man and can be found at http://senzee-stories.blogspot.com/. Moved La Casa de los Ancianos, Future Headlines and Language Vector over there. If you can't find them here anymore, now you know why.

Did a little pranking at work, as is the custom when a coworker leaves his desk unattended while away on a vacation. Report on that coming shortly.

Other stuff going on too. Can't really go into much detail about it at the moment.

Which Languages Handle Packages and Libraries Best?

2008-04-11T15:32:00.003-04:00

In the post Why Do We Use C++, I made the following statement.

Another, often forgotten strength of C++ and of many traditional modular and modular-turned-OO languages is linking, or more generally, we might say 'package management'. C++ innately offers build-time linking and runtime 'linking' is also usually available (i.e., DLLs). This allows graceful scaling. Tool support for this in C++ is strong and reasonably robust because C++ is so frequently used to build enormous projects. It's possible to build massive projects in pieces in ways that are awkward or impossible in many functional or logic languages.

Simon wrote,

C++ good at linking and package management??? That's the most ridiculous thing I've heard all week. This is one of the weakest aspects of C++. Where to start:
* no standard ABI for compiled code. ("Don't mix compilers")
* no standard for managing namespaces and packages like in Java.
* Very hard to evolve a class and keep binary compatibility. Lots of obscure compiler depend rules need to be followed to do this. ("Change a .h file, rebuild everything that uses it")
* dynamically loading code modules has to be done manually. Java, Python and other languages load classes on demand.
* C++ code can call C code, but any other kind of mixed language development is a nightmare.
* easy to break features like dynamic_cast when loading share libraries. (My fun debugging adventure this last week...)

I think Simon makes some excellent points here. So I'd like to ask the community.

What is the best language (and what are the best language features) for linking/loading and package and library management?

Social Networks Could Still Change The World

2008-04-10T20:46:00.023-04:00

Social networks are underwhelming. Join one and add your friends. Perhaps they are friends you haven't seen in years. You'll get excited, exchange a message or two and then forget about it.

One social networking site is better than many.

I have circles of friends in MySpace (a circle of 0), Facebook, LinkedIn, Classmates.com and Reunion.com. The categorization is artificial. I'm friends with people I know who also happen to be members of the respective site. Take Classmates.com for example. My main group is anyone who attended my school the years I did. Thinking honestly back to high school, there are people I liked, people I don't remember and a dozen levels of nuance in between.

Classmates.com and Reunion.com are dead. They are made utterly redundant by Facebook. LinkedIn is a better alternative to exchanging business cards but could easily be swallowed up by a site with a more flexible, natural and/or automatic way of specifying circles of friends and/or privacy policies. MySpace, on the other hand, just sucks.

Until social networks get wise to the psychology of social interaction, they won't be really useful.

I feel uncomfortable mixing my different circles of friends at parties without making sure that they can make some sort of connection. Categorizing relationships is something that we do effortlessly and unconsciously. Determining what to share with who, when and where is a complex bit of emotional processing that either comes naturally or not at all. For a social network to be really useful, it's going to have figure this out without us having to explicitly group all our friends. After all, these groups - especially the heavily political circles (I cringe for the oceans of middle and high school students on these sites) - are transient with shifting boundaries. You may say that asking a machine to understand this is asking too much, but I disagree. Given enough information - for example the phone logs in your mobile phone - and the behaviors of everyone else who has ever used a social networking site, existing and proven statistical algorithms such as Amazon's book recommendation could automate the largest part of this (Note that statistical algorithm these days is a politically correct euphemism for AI).

I don't want a site. I want a social widget that I can drop into and enrich any technology I want.

The idea of going to an actual website to share anything with friends is absurd. What's more absurd is having to go to five of them. I need a social network component that I can drop into Madden, my phone, my emailer, my IM application, my videogames, my browser, my newsreader, my MMOGs, my iPod, my television, my casual games.

Would this be web component like the Google search bar at the top of every site? Under the hood, maybe I'd expect a sort of service or protocol like LDAP (social://facebook.com?paul_senzee) or a central social network database with intelligence. Of course, this implies that..

Whether one or multiple social networking companies prevail, there must be a standard data interchange format for social network information.

Companies that maintain databases of files of this format will learn a lot about who we are and..

We are known by the company we keep.

The individual as a social being implies that certain types of personal information are best and unavoidably contained in a social network. Then why not keep all of it there? Experience has shown that people are willing to forgo certain privacies for much greater benefit and it's easier to hold one company accountable for privacy violations than many. In this way, social networks could allow advertisers and vendors to present me with exactly what I want without having to give my information to 1,000,001 different parties or revealing my identity.

I want a web I'm interested in, and that my friends are interested in. Social networks can be great active filters.

In 1995, the idea of posting a website and speaking to an eagerly listening world was exhilarating. Now it's noise. I want things I'm interested in to come find me, because there's no way I can sort through the enormous mess of the internet efficiently. The social network knows the interests I share with each friend.

I'll find a story or site that interests me and tag it with a button in the browser. That percolates to my friends with similiar interests, not to the world at large, and not to my friends that are uninterested in the topic. If I'm playing a 360 game and I get stuck at a spot it would be cool if the game would show me how my friend passed that spot.

We have one global web. How about a billion personal webs filled with information actually relevant and interesting to oneself and one's social circles. A much, much more useful internet. Of course, I'll always want to be able to opt back into anonymity.

We need to publish information with the sensitive parts [deleted]. We want to publish a highly targeted message.

Imagine that we could share information with subtle variation based on the group of people that we're sharing it. If advertisers can do this to us all day long, why can't we all?

A new blog post..

After [show for="EA Employees" alt="our new game"]The Beautiful Princess Tea Party[/show] comes out[show for="EA Employees"] on 09/27/2011[/show], I'm going to take a long vacation in the Keys[show for="Senzee Family"] at the Gardens Hotel[/show].

For EA Employees it says,

After The Beautiful Princess Tea Party comes out on 09/27/2011, I'm going to take a long vacation in the Keys.

For my family it says,

After our new game comes out, I'm going to take a long vacation in the Keys at the Gardens Hotel.

See, The Beautiful Princess Tea Party is going to be a big surprise for my girls. Finally, for everyone else it says,

After our new game comes out, I'm going to take a long vacation in the Keys.

In this case, blogger.com would query the social network to process this page before it's served up. As you can see, to be really, really effective the social network must be an organized, standardized service at the internet's foundation, instead of a handful of ad-hoc websites thrown up to cash in on the Web 2.0 rush.

That's the web I see coming.

Philadelphia 1961

2008-04-03T22:38:00.004-04:00

(Philadelphia 1961, Lee Friedlander)

I saw this at the San Francisco Museum of Modern Art, and it struck a chord. This, ladies and gentleman, was the future.

Future Headlines

2008-03-19T12:58:00.018-04:00

Student Discovers O(log log N) Prime Factorization Algorithm | MIT Student Found Dead | Intel Delivers 128 Core Processor | Protein Folding Fully Modeled | Human Brain Functionally Described | Half of World's Population Found on the Net | Wireless Cortical Device | Computational Model of White Blood Cell Complete | Games Jack In | Immortal! Gene Therapy Makes Wealthy, Healthy Immortal | Computational Model of Human Body Complete | New US Childbirth Policy Follows EU

Just so you know, the below stories are fictional..

Student Discovers O(log log N) Prime Factorization Algorithm

COMPUTER SCIENCE NEWS (March 19, 2009) - Brad Halloway, an MIT graduate student, claims to have discovered a prime factorization algorithm that has a guaranteed complexity of O(log log N) where N is the number to be factored (or O(log N) where N is the number of digits in the number to be factored). Researchers are eager to see the algorithm, which Halloway will present in person in November. A number of researchers have provided him with several million-digit numbers, all of which are products of two very large prime numbers. Halloway claims that he was able to factor each of the numbers in about one week. The researchers verified that the factorizations were correct. "If this is true, there will be earth-shattering consequences for computer security .. (more)

MIT Student Found Dead

NBC (March 22, 2009) - An MIT student, Brad Halloway, was found dead in his apartment today, victim of an apparent robbery. The perpetrators carried off a computer and a video game console. This is the latest in a recent string of so-called "Video-Game Burglaries" around the campus. "This is the first where someone has died - he must have surprised the perpetrator," said Capt. .. (more)

Intel Delivers 128 Core Processor

PRESS RELEASE (November 6, 2013) - Today Intel Corporation ships the first of its 3.2GHz 128-core processors manufactured with a new 16 nanometer process pioneered .. (more)

Protein Folding Fully Modeled

SLASHDOT (December 6, 2014) - Thanks in part to the Folding @ Home project, researchers announced today that the long-outstanding problem of modeling the way proteins fold has been solved. .. (more)

Human Brain Functionally Described

SLASHDOT (April 19, 2015) -Scientists today unveiled a functional model of a human brain running on one of the world's most powerful supercomputers .. (more)

Half of World's Population Found on the Net

CNN (January 1, 2016) - It is now possible to find more than fifty-percent of the world's population on the internet. .. (more)

Wireless Cortical Device Demonstrated

MSN (August 12, 2017) - The Cortico device announced this week provides a complete-sensory brain interface to its recipient. Cortical Labs also introduced the surgical equipment necessary to implant the interface. Cortical states that the equipment is self-sufficient and requires no human surgeon to oversee the procedure. .. (more)

Computational Model of White Blood Cell Complete

JOURNAL OF COMPUTATIONAL BIOLOGY (July 12, 2019) - ".. this is a monumental day for computational pharmacology .." (more)

Games Jack In

TIME (June 23, 2020) - A new generation of games are set to hook directly into several leading cortical interfaces for an experience that's "far more reality than virtual," .. (more)

Immortal! Gene Therapy Makes Wealthy, Healthy Immortal

USA TODAY (May 23, 2026) Front Page - It is a move that critics compare to the rich trampling the poor in a rush to leave the Titanic. Yesterday, the FDA approved gene therapy for rendering those (very few) who can afford it - and are in good health - biologically immortal. The subject of intense controversy since deemed technically possible only seven years ago, it has only now .. (more)

Computational Model of Human Body Complete

JOURNAL OF COMPUTATIONAL BIOLOGY (October 3, 2029) - ".. we've obtained the Holy Grail of computational pharmacology .." (more)

New US Childbirth Policy Follows EU

USA TODAY (February 8, 2040) - Following in the footsteps of the EU, the United States today introduced a childbirth policy that requires government permission to bear a child .. (more)

.. and this list is subject to change without notice. ;)

Technology Is Disappearing

2008-03-10T21:46:00.006-04:00

When I was a kid I knew what the future would be. It had the clinical austerity of 2001: A Space Odyssey, the cold silver cities of Logan's Run and the dark, bleak dirtiness of Blade Runner and Alien. That future was inevitable. More optimistically the colorful, gaudy, in-your-face future of Back to the Future II came along with a number of films of the late 80's. Everything in the future was going to be complex. Every surface would have a thousand buttons.

The 90's began a different trend in science-fiction. In one episode of Star Trek: The Next Generation, Picard returned to a modest country home in France that could have come from the 1950's - or the 1850's. Gattaca and Minority Report contrasted simply and starkly adorned futures with warm, familiar early 20th century fashion and interior decor.

The later examples are a truer glimpse of the future. Nobody wants to live in a world that's metallic, austere, cold, dirty or gaudy. Developers of technology in our world strive for naturalness, unobtrusiveness and basic elegance in design. The less visible, the better.

Consider the ubiquitous computer. It's so pervasive that even talking about 'computers' seems anachronistic. In the 80's people would say about an IT professional, "he works with computers." Now they say, "he's a game developer," "he's a graphics programmer," "he's a web designer," or "he works in information technology." And everyone knows what those things mean. Back in the 60's and 70's, computers were novelties, understood by few. A computer was a sure sign of the future. Today, we spend lives sitting in front of them. We know them for what they are - tools of immense and perpetually untapped power that also happen to be phenomenal pains in the ass. What is the future we hope for now? It's a future where we can reap the benefits of machines and be free of the frustrations they cause us. So a computer doesn't look like the future anymore, it looks like the frustrating present. And, if you draw too much attention to it, it starts to look like the past.

What does this mean? Technology must disappear. No, it will not go away, but it will become invisible. Take the beautiful iPhone. All those complicated little buttons you find on a smart phone? They just went away. It could only be better if it wasn't there at all, while still doing all the iPhone allows. Research proceeds rapidly in flexible electronics, stretchable electronics, wearable computers, transparent electronics, nanotechnology and most significantly, brain-machine interface, so this may come sooner than we think.

PlayStation 3 Cell Came Before Its Time

2008-03-05T19:22:00.018-05:00

The Microsoft Xbox 360 platform is now profitable and Sony's PlayStation 3 is gaining parity with the Xbox 360 in the marketplace. We're about halfway through this console cycle (these cycles typically last about five years) and we can reflect on technology choices with the benefit of some hindsight. The ramifications of some of the more obvious choices, for example PS3's Blu-Ray vs. 360's plain-old-DVD, are well known. Microsoft's background as a operating system and compiler developer provides the 360 with perhaps the best development tools ever to grace a non-desktop system. We'll look at that in another entry. We'll also want to compare the Xbox 360 GPU and the PlayStation 3 GPU.

For now, however, I'd like to look at the Cell. In doing so, I'd like to ask and try to answer the question, "Why has the quality of PlayStation 3 titles lagged behind that of the Xbox 360 titles?"

The Cell Processor is Amazing

From the info I gathered, the PS3's Cell should have a peak numerical performance of about 205 gigaflops. From Wikipedia, I gather that the 360, on the other hand, has peak performance of 116 gigaflops. The PS3's Cell has one regular PowerPC core (with two hardware threads) called the PPU. It also has seven SPUs (one is reserved for the OS), which are streamlined vector processors that are fast as hell, although functionally limited compared to the main PowerPC core. They have their own memory at 256K apiece. Half of that is available to the developer. The 360's Xenon processor, on the other hand, consists of three conventional PowerPC cores almost identical to the Cell's PowerPC core, for a total of six hardware threads. So, if you're like most people, you're probably thinking, "Wow, the PS3 blows the 360 away!"

Except that it doesn't.

Developing Games is Hard

I just read this at .mischief.mayhem.soap about the size of the Assassin's Creed team. Actually, I think that's about the same for Madden NFL. Modern games require an enormous amount of content, some of which must be reflected in code. Games on multicore consoles are much more difficult to debug. Making games is simply much more expensive that it used to be. I heard a colleague remark that he saw that one PlayStation 1 version of Madden had 30 files. Now, it's thousands of files, millions of lines and scores of developers.

Developing for the Cell is Hard

When we first saw the specs for the Cell, I thought, "it looks like it was designed to play movies." And in playing movies, the PS3 will blow the 360 away. The Cell was designed for performing the same kind of vector calculation on large numbers of small chunks of data. Essentially like a GPU, but more flexible although less parallel. There's a lot of this sort of thing in games. Some physics, audio decoding/encoding, movie decoding/encoding, some graphics/geometry processing. Still, there are a lot of things that aren't well suited for this model. Among those would be language interpretation (we do a lot), gameplay code, sparse data manipulation, and a lot of really mundane and common software engineering things that are inherently sequential. Sorting a list of pointers that must be dereferenced to be compared, for example.

Explanations of why the 360 performs so well compared to the PS3 usually start out like, "Well, the Cell looks better on paper, but.." Developing for a 360 is like developing for a multicore PC. Take your existing code, multithread it and know that it'll work on the 360 and upcoming PC-like multicore machines with little changes.

You can build PS3 software exactly the same way using only the PPU and its two threads, but that will yield almost exactly one-third of the power of the 360's processor. In fact some early titles that were developed first for the 360 did just this. This was not as bad as it sounds though. A lot of early code for these machines was not thoroughly multithreaded because much of that code was directly ported from older Xbox and PS2 code. In these cases, it used just a portion of the 360's power anyway.

Now we're pushing the capabilities of the 360 and trying hard to get the PS3's six fast but special-purpose SPUs to do what we can do with the four remaining general-purpose hardware threads on the 360. It's hard, though certainly possible to achieve parity. Still, it requires a lot more code and expertise just for the PS3 (usually from our best developers), which typically means much less overall developer productivity.

The Cell Looks Like the Future

Unfortunately, as delightful as it is to develop for, the approach taken with the 360's Xenon processor simply can't scale. At some point, with some number of traditional cores, memory access and synchronization are going to break down or become so contentious that any benefit of a larger number of cores will be nil. So the Cell looks like the future. A small number of traditional cores with a large number of stream processors can yield phenomenal performance in the right proportions. I predict that some member(s) of the next console generation will have a processor with 8 traditional cores and perhaps 256+ stream processors. 256 sound ridiculous? Consider that an ATI HD 2900 XT GPU has 320 stream processors.

Why do I think the Cell was premature? It did not fulfill the possibilities of traditional multicore processor before moving on to a hard-to-develop-for, though potentially more powerful, architecture.

In any event, it's clearly a foreshadow of processor architecture to come. Still, the difficulty of achieving parity between the Cell and 360 Xenon when game development is already at its most taxing really isn't doing the PlayStation 3 any favors.

Simple Templater in C++

2008-03-04T22:47:00.007-05:00

Like I mentioned before, I'm posting some of my personal code library that I've accumulated over the years. If you modify or improve this, send the changes back please.

Recursive template engine with escaping and lazy evaluation. Note that it is case SENSITIVE.

Use
1. Add items to Templater's map.
templater.map()["bob"] = "joe";

2. Evaluate a text item that references a map item.
s = templater.eval("well, hello [!bob]");
Returns "well, hello joe" in s;

Code
templater.h
evaluator.h
evaluator.cpp

Introversion's City Generation

2008-03-04T21:32:00.008-05:00

At GDC I came across Chris Delay from Introversion Software. I recognized him from the picture next to his blog. I've been admiring the work that he's has been doing with city generation and I was happily surprised to meet him in the line for one of the sessions. Not thinking, I extended my hand. I hope I didn't get him sick.

I asked him a ton of questions about it, 'cause I'm working on similar ideas. Anyhow, please check out the city generation stuff he's doing. He's working from Muller and Wonka's shape-grammar/L-system generation ideas and producing some awesome work.

Check out this beauty -

Emotiv Systems

2008-03-04T20:22:00.015-05:00

While at the Game Developers' Conference, I checked out Emotiv Systems and their EEG-based input ('mind-reading') technology in the form of the $300 EPOC to be released later this year. I spoke to the Emotiv guys at their booth, they seemed a bit secretive about their technology. They described it as telepathy. The EEG input device concept is interesting to me, but seems a little gimmicky. In fairness, though, I didn't try it. Like a resurrected computer-enhanced biofeedback machine? Not sure how well it would work in real life. The devices don't use conductive gel like real EEGs use. The input device also tracks eye and facial muscular movement, so it's more than just an EEG, but an EMG as well. Also, EEGs require the user to be still. That's a tall order for a videogame player. Anyhow, assuming that sophisticated signal processing could overcome the lack of conductive gel and motion, it seems that the response time in changing gross brain state (for example, to punch a bad guy) would be glacial and using it would be exhausting. What is really interesting to me, though, is how they're doing their signal processing. My initial guess is principle components analysis with a prototype database. I wonder if I'm way off base there.

An article about Emotiv's brain gear.

Down and Out on Market Street

2008-03-04T19:46:00.014-05:00

As I alluded to in the Fable 2 post, EA sent me to the Game Developers' Conference (GDC) this year in San Francisco. Unfortunately, I saw far less of it than I would have liked because I was down with the flu. I think I spent more time in bed and at the Walgreens at Market Street & 4th than I did at the conference. That may be an exaggeration, but it was still disappointing. I also ended up getting Colin Campbell (a colleague of mine) all sick, which just sucked doubly.

Networking. No. It's hard to schmooze when you're coughing every 12.2 seconds and trying to keep your nose from dripping on your new friend. So I didn't meet a lot of the people I'd hoped to. Who I did meet, I tried to keep my distance and avoid shaking hands so as not to pass on the scourge. Still, it wasn't a total loss. It was the first time I'd been and it was fascinating. The keynotes were great, including a compelling Ray Kurzweil. I didn't feel well enough to focus enough on the technical sessions to get much out of them. Still, there were a lot of ideas in the air. I also attended several game design and art sessions.

Need Input

The WiiMote and Guitar Hero/Rock Band must have spawned a lot of thought about input devices in the industry. Depth cameras and a bunch of other technologies were on display. We had NeuroSky and Emotiv Systems with their bio-sensor technologies (see the following post on Emotiv Systems).

Physics Engines, UI and Procedural Texture Technologies

A lot of middleware vendors especially surrounding Physics and UI. For physics, we had the Octave Engine and others. For UI - Scaleform, Anark, MenusMaster. Scaleform seems to be quite popular and is a Flash-based UI system. I was taken by Allegorithmic's procedural texture technologies. Oh, and motion capture.

BioShock

Everyone was talking about BioShock. I've not yet started playing it, although I do own it. I'm still playing Assassin's Creed right now which is phenomenal in its own right. I'm excited to get at it. I attended a session with Ken Levine, BioShock's designer.

More

PC World Video, Recap
GDC08 Review

I already wrote about Fable 2, though I'll probably amend that (and this) entry as I get more time and reflect a bit more about it.

Java-Style Properties Files in C++

2008-02-28T20:00:00.007-05:00

Need to handle Java-style properties files in C++? I've decided to post some of my own personal library of code on this blog. This is one of those occasionally useful things.

propertyutil.h
propertyutil.cpp

If you fix any bugs, extend the code or anything else, send back your changes if you don't mind. ;)

Fable 2 News (GDC08 Preview)

2008-02-25T08:51:00.015-05:00

In San Francisco last week (more on that later), I heard Peter Molyneux talk about Fable 2. See, I love the first Fable, and I'm really excited about this new arrival.

He announced four new features which are on Wikipedia already, so for many of the details you'll want to head there. As a part of the demo, Peter demonstrated his onscreen counterpart, The Molynator who, like a solid 10% of the audience, was female. Fable 1 had just one possible (male) protagonist. He announced that The Molynator married (after meeting at a courting spot, or something like that) five years ago. They'd found a home in the slums of Bowerstone early on, then moved to a country residence a few years later. The young couple had a young child in that time. Yes, Molyneux said, she experienced all the joys of virtual pregnancy, including the growing belly.

Pregnancy

He said he didn't know if the big belly would actually make it in the game. Something about a nine-month pregnant woman slicing open a dragon or troll with a broadsword strikes me a little inauthentic. Especially if you've ever seen (or been) a nine-month pregnant woman trying to get out of a chair. Also, might it be bad form to have a pregnant woman defeated in battle?

Imagine you choose to have a family in Fable 2. Molyneux strongly hinted that your family will become a part of the narrative. He glossed over this one really fast, though. Your children, your spouse kidnapped, murdered? Avenged? Who knows?

Co-op play inspired by LEGO Star Wars, but much more dynamic. Rebekka and I can play Fable 2 together! So now we won't have to worry about her getting that big belly again. ;) Well, until we finish it. He didn't say anything about the length. Hope it's longer than Fable 1.

There's no overhead map. Instead, there's an innovative breadcrumb trail showing you where you've been most.

Check this out. No money for completing quests? Arghh. You either have to gamble or be hired as a henchman to make money. That just doesn't make any sense to me. I have to get a job to make money? That's way too much like the Sims. Which means way too much like real life. I'm not much of a gambler either. The weird thing is I already have a job. It's the one that I'll use to pay for Fable 2. And I like it more than gambling or being a henchman.

One of the gambling games, Keystone - a roulette like casino game - will be coming onto Xbox Live prior to the release of Fable 2 so that you can make your money before you need to spend it.

I'm excited to play it. I can wait though. I've still got Assassin's Creed and BioShock to work through (not necessarily in that order), so The Molynator's got plenty of time.

More info on the presentation.

They Say Games Are No Good For Kids...

2008-02-11T17:34:00.001-05:00

This article (Xbox is Crack for Kids) is kind of funny in a way. This offhand opinion met with hundreds of angry replies. Honestly, I don't think that the game-playing community needs to be so defensive. Games, or more broadly, interactive-everything, is the future.

That article represents the solitary voice screaming at the tsunami to keep its distance. We humans want to be entertained. We go to great lengths to build entertainment technology and we've always adapted non-entertainment technology to serve the purposes of entertainment.

In any event, I've long held that good-quality games are excellent teaching tools. My eight year-old son is a strong reader. He's always loved games and has (coincidentally) been drawn to games that require heavy reading. Nintendo fans know what I'm talking about. Because of the games, he was motivated early on to read, and the interactivity of games pushed him further in that direction. Now, paradoxically, he loves reading novels (but still plays games!).

Here's another example. My children love Tim Schafer's phenomenal Psychonauts. About a week ago, while getting the kids to sleep, I found my six year-old daughter sitting on her bed deep in thought.

"What are you thinking about?" I asked.

"Daddy, why did Napoleon lose at Waterloo?" She asked.

I laughed, "I don't really know, but we can look it up.."

I don't think her first-grade textbooks could have provoked that question.

Xbox 360, PS3 Games Unplayable on Future Hardware?

2007-12-30T12:32:00.001-05:00

Earlier I posted about concurrency being a crucial issue in today's and tomorrow's games.

Recently, my son has been playing the original Xbox game Thrillville. The game is solid on our original Xbox, but crashes with disturbing frequency on the Xbox 360, particularly during loading. This leads me to suspect that the timing differences between the original Xbox and its emulation on the 360 exposed threading issues that were present but simply did not occur because of the timing characteristics of the original console. Console game developers generally test exclusively for the single configuration per target platform. The only way we find threading issues, it seems, is through testing. So if testing fails to reveal real issues, games are shipped with them. This is mitigated, of course, by developing games for multiple multithreaded platforms (ie, Xbox 360 and PlayStation 3) with the same code base.

Emulating Concurrent Code

This is one kink in a larger problem. The problem is developing a body of 'literature,' whether games or other software, that will be usable on future machines. Take, for example, the classic game Ultima VII. It is a beautiful and vastly influential role-playing game rendered unplayable by modern machines because of its hardware voodoo. (We also may occasionally need to use software such as old versions of WordPerfect, VisiCalc or Lotus 1-2-3 - and perhaps from non-PC machines - to read crucial, but old, data stored in those formats.) For such situations we typically resort to emulators such as the excellent DOSBox or AppleWin for maximum compatibility.

It is very difficult, if not impossible, to maintain timing-level compatibility in emulators. Older games that relied heavily upon machine-specific timing are now pretty much broken in emulators. For that matter, they're generally broken when the next generation of hardware arrives. Multithreaded code that contains hidden bugs (which is pretty much all code, multithreaded or otherwise) is implicitly and heavily timing-dependent. It's one thing to emulate an Xbox game, with typically very low levels of multithreading on an Xbox 360, and may be quite another to emulate Xbox 360 games on an Xbox 720.

As a result, I suspect that far more Xbox 360 and PlayStation 3 games will be unplayable on future hardware than Xbox and PS2 games on the 360 and PS3.

Why Do We Use C++?

2007-12-20T10:04:00.004-05:00

One of the recurring questions (especially when confronted with concurrency) is 'why do we use C++ for game development?' This entry was extracted from Concurrency in Game Development since it's really a separate topic all its own and that one was overlong anyway.

Why Do We Use C++?

Now before I hear the platitudes about how [insert favorite non-C++ language here] is better anyhow and would fix everything in one shot, let me ask you how feasible it would be to develop multi-million line software on multiple, often brand-new, platforms simultaneously with extreme performance expectations, leveraging mega/gigabytes of shared legacy code and without having to port, develop and/or maintain one's own compilers and toolsets with this superior language. If the game development industry turns to non-C++ languages to solve this, it will be slowly and painfully. And, most likely, it will be an industry-wide move and not just one studio or another, although some must lead the way. Honestly, I would never expect C++ to go away completely (just as assembly has never really gone away), and some future approaches may just be layered on top of C++ (Bigloo, Intel Ct, OpenMP), or use it when performance is critical (Java's JNI).

C++ is flexible and fast. With sufficient (possibly enormous) effort, it can do almost everything any language can do. It can function both as a high-level multiparadigm language and as a low-level portable assembly language. What makes C++ (and C) so widespread is its unrestrictive nature. This is widely seen as a negative, but in the real world, being able to abuse your language to get what you want from your machine is of crucial importance - especially when performance is a primary concern. That said, you may only want to abuse your language say 2-5% of the time. The rest (95-98%) of the time, you'd like some nice, type-safe, bounds checking, memory-managed, interactive, memoizing language, giving you a >100% increase in productivity. That we have settled on C++ indicates that the 2-5% is so crucial that we're willing to sacrifice the rest for it. In the game industry, I think that's a fair statement.

Another, often forgotten strength of C++ and of many traditional modular and modular-turned-OO languages is linking, or more generally, we might say 'package management'. C++ innately offers build-time linking and runtime 'linking' is also usually available (i.e., DLLs). This allows graceful scaling. Tool support for this in C++ is strong and reasonably robust because C++ is so frequently used to build enormous projects. It's possible to build massive projects in pieces in ways that are awkward or impossible in many functional or logic languages. (See comments, some strong points about other languages package handling vs C++'s and Which Languages Handle Packages and Libraries Best?)

Still, writing solid C++ code even in the absence of multithreading requires a mastery of nearly the whole language, making it dangerous for inexperienced developers. Even merely adequate C++ coding is heavily reliant on learned idioms that are not a part of the language, and therefore unenforceable. For example, that you are allowed to return a pointer or reference to a stack-allocated object from a function, or that you are allowed to overflow a string buffer have cost the world untold man-hours and dollars. And yet for all its ills, C++ ranks near the top of the most useful (or at least used) languages ever explicitly designed (including Esperanto).

(4/11/08 - adapted from one of my comments in reddit.)

I should stress again that one of the more crucial issues surrounding language choice is the set of tools provided to us by the console vendors. C++ wasn't adopted until late in the console world because C++ standards weren't well supported by console vendors. Tools have always been poor on consoles compared to the PC (this has changed somewhat with the 360), and good quality C++ compilers were rare in previous console generations, but C compilers were available (athough they, too, came late - with the original PlayStation or Saturn, I believe?).

The PC gaming world hasn't seen this sort of lag. Game developers are eager to adopt new technologies. Other technologies have been unavailable on these platforms.

This is an exploratory post (as they all are..) and is subject to change.

Concurrency in Game Development

2007-12-18T11:13:00.001-05:00

Effectively developing concurrent software for modern machines with multiple cores is perhaps the greatest technical challenge we'll encounter in some time. Our current approaches just aren't up to the task of creating robust multithreaded code.

Why Do We Use C++?

Please see the linked post for some thoughts on this topic. Before we look directly at concurrency, let's look at what brought us to where we are.

Categories

It’s reasonably well established that categorization is a fundamental human strength. You might say that feature extraction and classification are hardware-accelerated in the brain. Even at levels far below the conscious (i.e., the visual cortex) information is categorized before it even becomes 'thought' to us. In fact, categorization is so fundamental to human thought that assuming categories themselves are real objects has been a universal illusion. In “How the Mind Works”, Pinker states that nearly all cultures initially adopt a ‘folk idealism’ as a result of this. Applying this to the boundary of man and machine communication, Object Orientation has evolved as a straightforward way to map categories (type, class, etc.) and instances of those categories onto machine architectures that deal primarily in a few primitive and largely undifferentiated numerical types.

Abstractions are complexes of categories and their interactions. Understanding complexity in terms of hierarchies of abstraction is something that people do really well. Modular Programming and, to a greater extent, Object Orientation attempt to give us tools to work at these levels of human competence - with, of course, some consequences in terms of final performance.

The predominant programming paradigms attempt to map the way people think onto the way machines operate. Let's turn our attention to concurrency and see if we can stretch this a bit further.

Concurrency as Time vs. Space

In software development, one of the things I’ve noticed is that people are much better at understanding space than time. This is why we map out time in timelines, MS Project files and a million other ways in spatial form. What this means to programming is that anywhere you can map out state in terms of space, the result is far easier to comprehend. For a clear example of this, see Google's MapReduce.

This is the core issue with concurrency. It’s very difficult to understand the possible states of concurrent systems because they happen in time. Many of the abstractions that can help with concurrency (such as Functional Programming, Message Passing, etc.) are useful because they essentially transform a state-heavy process into some equivalent but more understandable spatial map whose design appears much more static.

Open Questions

In some ways the game industry is at the vanguard of multicore development on consumer machines with the Xbox 360 and the PlayStation 3. The amount of time and effort that we spend finding and fixing multithreading bugs is terrifying. While the hardware companies move toward doubling the number of cores with each processor generation, software companies will be reeling.

Clearly, C++, as it is currently, is not well suited toward developing software on highly multicore machines. Ideally, we’d have a language, extension or paradigm that would allow us to map thread concurrency onto easily understood (that is, spatial) language constructs that discourage or prohibit the kind of multithreading errors that currently waste hours and hours of our time. Right now, we don’t have to worry how many registers there are on the processor when we write C++ code. Similarly, whatever language/paradigm we’d want to use would, at the compile stage, optimally generate code for the number of cores available to it on the target platform.

So would some sort of functional language be best? C++ with functional extensions? Erlang? Haskell?

What about programming models based more on hardware description languages such as VHDL and Verilog, which are inherently spatial? Would they map more effectively to multicore machinery since hardware languages describe processes which are inherently asynchronous?

In any event, we software developers have an interesting road ahead.

Times of Lore (maps)

2007-12-12T22:58:00.002-05:00

A few years ago I wrote a program to spit out the maps from the PC version of the 1988 game Times of Lore, which I loved as a youngster. Where there were cities mostly show up here as bright green grassy areas. Thanks to Andrew Schultz and his site for inspiring me to dig up and post these images. Oh, and check out Kenn Rice's site too.

The world -

The dungeons -

The Impossible Dream #4

2007-11-09T23:37:00.002-05:00

(Updated 11/23/2007)

Where Have I Been?

It's been months since I've blogged. They've been productive months, though. See, in early October, another beautiful baby girl joined our family! It's been a rush having another child. It's incredible how you can still be completely awestruck as if it were the first time, each and every time. :)

Prior to that, vacation.

Because I had some time off, I slipped in some hours on my Impossible Dream. I hadn't really considered blogging about it yet until I stumbled across Project Steel & Glass at The Cluttered Desk that motivated me to write an update.

Before leaving on vacation this summer, I'd made some headway with Pair, my embedded script language. Especially, I'd made some strides compiling it to more efficent bytecode, etc. It's in a sort-of usable state. Still, some things like lexical closures are not 100%. I'll leave blogging about that until later.

Basically, much of the material pass (no illumination yet) of the game is rendering. It looks pretty raw and repetitive with respect to content right now. There's not yet much differentiation in city sections. No niceties such as shadows or even basic lighting yet. And, jeez, how am I gonna do trees? It's time, though, to switch gears for a bit and prototype the gameplay. As you might infer from the screenshots, the protagonist will be capable of flight. I've got my Xbox 360 controllers hooked up to my PC and I can fly all around town.

(Zaragosa, Mexico [fictional])

(Puebla, Mexico [real])

Please, I'm Just One Man!

So I must enlist the machine to generate a great deal of content automatically, albeit offline. This approach, of course, runs the risk of a high degree of repetition, but I think with some caution and tool refinement this will turn out great. I'm really happy with how auto-generation has gone so far. It's actually starting to vaguely resemble the residential city streets of central Mexico, which is, of course, the idea.

Anyhow, I'm pretty pumped about it. Given some time (lots) and little-by-little refinement, we may have something pretty cool here.

NCAA 08's Video Highlights

2007-07-31T21:19:00.007-04:00

I'm pretty proud of this one. ;)

Thousands of user-created videos from NCAA Football 08!

http://www.easportsworld.com/#canvas_TopVideos

I posted this on 4/12/07:

I've been working on a little something special in NCAA Football 08. An NCAA Football 08 First Look article by GameSpot details the surprise. So now that it's public..

".. Once you've found a highlight you're particularly fond of, you can then choose to create a highlight of that play. This lets you choose from multiple camera angles to take a snapshot or segment of game-generated video. From there, you can either save these photos or videos on your hard drive or upload them to share with friends. .."

About the middle of last year, I was asked to do some R&D for this technology. Not long after that we were building it. Still, it came really close to being cut on a number of occasions. There were times when it seemed impossible to satisfy all of the below constraints and still produce a video of acceptable (or any) quality. I had taken some time off for a bit of surgery as NCAA was finalling and came back to find some bugs in my .flv encoder. So, I was a little paranoid that some random, unforseen issue might render the whole feature unusable in the shipped game.

Constraints

There were some wicked constraints putting this one together including -

Encoding while rendering the game at 60fps/X360 and 30fps/PS3 without dropping frames.
Keeping final file size small enough to not overwhelm the servers and to keep our bandwidth costs down.
Final visual quality given the other constraints.
Memory during encoding. Early in the cycle, I gave a rough estimate of 25MB free memory in-game to encode to .flv. It turns out we had to do it in under 500K in-game, and we did that by offloading some tasks until later and some other tricks I won't - and shouldn't - go into ;).

Compromises

No sound. This one's hard to take but we just didn't have time. Remarkably, on the web, I haven't seen anyone complain about this. For next year.
15 frames per second. Yes, we wanted thirty, but we could only do that with a smaller frame size. Believe me, the trade off was well worth it. Also for next year.
Video dimensions. We would have liked a larger frame size but we just didn't have the machine resources or network bandwidth to spare.

All in all, I think we struck a reasonable balance and it works. It's exciting to see people this jazzed about it! This technology is now quite the buzz around EA and next year we'll be seeing it in a lot more titles.

Will this technology and its descendants usher in the Web 2.0 era in console sports gaming?

I hope so!

Update Jan 3, 2008

Check out #2 on this list at pastapadre.com.

.. Launching with the release of NCAA Football 08 [EA Sports World] was an immediate success as fans of the game utilized the screenshot and video uploading features and it became one of the most positively responded to new additions to any sports game in recent memory. ...

For video highlights and optimizations geared at getting our football games to 60fps on PS3 starting with NFL Tour, I was nominated for the 2007 Outstanding Achievement in Engineering award for EA Sports - Tiburon. :)

Trampolines

2007-07-29T23:24:00.002-04:00

Developing embedded languages for applications in the past, the most time consuming part always seemed to be wrapping the multitude of foreign language (ie, to/from C++) function calls of interest with script-compatible functions.

One of the goals of Pair is to make working with other languages - especially C/C++ - very easy. So here we'll take a different approach. This time we'll generate trampolines to call foreign functions (for example, from a DLL) and do the boxing/unboxing of values for Pair to use. Trampolining has a number of different meanings but many of them center around the idea of generating code at runtime to call a function with special requirements that aren't known until runtime. In this case, we're directly generating the machine code necessary to set up the stack frame and call a function using either the __cdecl (for the C runtime library) or __stdcall (for the Win32 SDK functions) calling conventions.

This allows us to do things like the following -

(let
  ((beep  
     (import "kernel32.dll" stdcall uint Beep   (uint uint)))
   (system
     (import "msvcrt.dll"   cdecl   uint system (string))))

(system "dir c:\\root")
(beep 500 1000))

C++ code -

nativecaller.h
nativecaller.cpp

Nearest Neighbor

2007-07-10T21:19:00.001-04:00

What's the fastest way to find the closest pair of three-dimensional points in a large set? That was the question posed for the most recent Hacker's Delight challenge.

We just released the results for Hacker's Delight #7: Nearest Neighbor. A little premature, it turns out, we misplaced a bunch of the entries. Once the remaining entries were found, I ran the tests before I came home tonight and Jim should be putting them up soon. It turns out that the missing entries don't really affect the outcome.

This is the third Hacker's Delight challenge I've blogged about. Challenge #6: 64 Choose 4 came from the post Let Me Count The Ways and, in full circle, Let Me Count The Ways (Part II) comes from the challenge. From Hacker's Delight #4: Primes came Sieve of Eratosthenes.

Alistair Milne from EA Montreal blew Nearest Neighbor out of the water with a blazingly fast and beautiful algorithm for finding the nearest two points in a cloud of points in three dimensional space. For comparison the very slow reference implementation is here. I wondered if I could get any more performance out of Alistair's algorithm and came up with an uglier but ~25% faster SIMD optimized version of his algorithm. Sadly, my own reasonably fast implementation for Nearest Neighbor is incorrect about 7% of the time.

My implementation, and many others we received were based on the canonical Nearest Neighbor algorithm as described in Cormen's Introduction to Algorithms. It seems that in every respect, Alistair's algorithm is superior. It's more elegant, it's much faster and it does less work.

Be sure to check it out.

A Developer's Life Stages

2007-06-30T10:32:00.002-04:00

I ended up writing a novel as a comment on Jamie Fristrom's blog post on Speculative Generality. Read his entry. I'm reposting my comment here.

Developing The Simplest Thing That Could Possibly Work

In Forrester Research's 10 Mistakes in Software Development, #3 is overscoping a solution and #9 is jumping into development without enough research, and we've all done it. :(

A friend consulted at a large transportation company in the Kansas City area. They'd been gearing up for a massive conversion from their existing AS/400 systems to a completely Java-based system. Remarkably, in the early stages of the project, they decided to wrap the entire JDK in their own custom framework and decreed that all developers use these wrappers exclusively. As a result of this, they obtained -

little or no added functionality,
more limited abstractions than the raw JDK,
minimally tested code underlying everything,
the requirement to train all incoming Java developers on their framework and,
worst of all, the obligation to maintain many more thousands of lines of code.

I've observed that developers tend to progress through a number of stages in their career. These are like the stages of grief: inescapable. This is because each stage produces a set of skills and philosophies that are eventually synthesized in the seasoned software developer.

The Underengineer

The underengineer is elated at his ability to get things done quickly and be productive, disdaining others (usually the overengineer) for what they see as excessive rumination and for producing an excessive amount of code for a given task. You can count on these guys to get stuff done. Will it be done 'the right way?' Only if by accident. Underengineers are often unable to discern 'the right way,' often because they've spent little time reading and maintaining other people's code.

A developer may begin to come out of this stage when he tires of fixing or writing the same code (in different places) over and over again. Or perhaps he'll be propelled into the second stage by an enthusiastic reading of Design Patterns or a sudden grasp of UML.

The developer at this stage is fascinated by accomplishing the task at hand and it is this stage in which he learns how to code and debug.

The Overengineer

The overengineer knows that he can solve any single development task and has become fascinated by possibility. Abstraction is magical. How can any piece of code be used to solve multiple tasks? Paradoxically, this usually ends up producing more code for any one task. While the underengineer shakes his head at this, the overengineer knows that his approach will produce much less code for an entire set of similar problems. Meanwhile, the overengineer looks at the underengineer's hackery with contempt. The question the overengineer rarely asks himself, though, is will there be a large enough set of similar problems to justify this effort? And so he abstracts everything in the name of flexibility.

The overengineer is sometimes shocked to realize he's become much less productive in terms of functionality than before. He can't seem to get anything done! Still, he produces as much (if not more) code and suddenly becomes capable of building much larger systems than he could as an underengineer.

This is the stage in which the developer learns architecture and the value (and painfully, the costs) of abstraction. This is the stage of the architects of the transportation system conversion discussed above.

What kills the overengineer? Maintenance. Dependencies. Broken abstraction. Realizing that the flexiblity he built in is unused and real systems change in ways he could never have anticipated. At some point, and probably many, the overengineer will get a phone call at 2am asking him to come in and fix an issue that breaks his whole architecture. After a few of these heartbreaking moments, his world view begins to change again.

The Seasoned Developer

The seasoned developer is skilled in coding and debugging from his time as an underengineer and is a skilled architect from his time as an overengineer. What's more, he's come to understand the following:

The primary reason for abstraction is to simplify implementation for a given set of requirements.

If abstraction doesn't simplify, ditch it.

The seasoned developer also realizes that the solution domain for a given problem may extend beyond technology. People often try to use technology to solve social or organizational problems. Conversely, people frequently attempt social or organizational solutions for essentially technological problems. Recognizing these incongruities before they become a software design and implementation can help to avoid doomed projects.

The organizational problem in the transportation example above is that software architects were assigned to do a job before anyone knew what job they were to do. The proper response would be to do nothing - or better, to work on a different project until the problem domain was fully understood.

Instead they chose speculatively general busy work, incurring a much higher cost.

The Future of Processor Hardware

2007-06-29T16:56:00.001-04:00

Many-Core Processors, Languages for Multiple Processors and Reconfigurable Computing

Many-Core Processors

Multicore processors are giving way to many-core processors with Intel estimating the number of cores doubling with each processor generation. The paradigms of the most commonly used programming languages are ineffective with respect to these types of processors. So what will happen?

Languages for Multiple Processors

Modified C++/Java# with futures, promises, (Flow Java)?

Functional languages are inherently more parallelizable than imperative languages. Will we see a resurgence of these types of languages?
- Concurrent Haskell/Concurrent Clean
- F#

High-Performance Reconfigurable Computing

Hybrid systems combining conventional (Von Neumann) architectures with FPGAs and other forms of reconfigurable hardware are starting to mature, but with radically different models of "software" development.

Is High-Performance Reconfigurable Computing the Next Supercomputing Paradigm?

From Wikipedia

Paradigm Shift

It is called the Reconfigurable Computing Paradox, that by software to configware migration (software to FPGA migration) speed-up factors by up to almost 4 orders of magnitude have been reported, as well as the reduction of electricity consumption by more than 1 order of magnitude - although the technological parameters of FPGAs are are behind the Gordon Moore curve by about 4 orders of magnitude, and the clock frequency is substantially lower than that of microprocessors. The reasons of this paradox are due to a paradigm shift, and are also partly explained by the Von Neumann syndrome.

EASTL

2007-06-26T18:12:00.001-04:00

EA genius and colleague Paul Pedriana published a paper to the C++ Standards Committee detailing his EASTL, an EA version of the Standard Template Library that provides a number of efficiencies with game development in mind, but that are nonetheless applicable across other software domains.

I've always been a fan of the STL in general, and since I've been at Electronic Arts, of EASTL. It's nice to see some of these innovations get out into the world.

Spread This Number?

2007-06-21T00:10:00.001-04:00

AACS encryption key controversy

This bad boy?

13,256,278,887,989,457,651,018,865,901,401,704,640

So where was I when all the noise about this was going on in April? Ah, illegal numbers. What will they think of next? Anything that can be represented digitally can be represented as a single (usually very large) number. Software, movies, music, etc..

That reminds me of a story I read once about a civilization that encoded all its existing knowledge as a single mark on a stick. The position of the mark on the stick divided by the length of the stick yielded a decimal number less than one. The digits of the fractional part of that number represented the totality of the civilization's knowledge in an encoded form.

Could this work?

Let's think about this. A proton has a diameter of about 1.5E−15 m. That means we'd get only about 16 decimal digits before we're measuring sub-subatomic dimensions. And Shannon tells us it would take lots of digits to represent that knowledge in any reasonable form. Actually, it would take about 2.3 million decimal digits per megabyte of data. And we're talking at least trillions of megabytes of data.

So no, there is no way - now or ever - that this could work.

Horrific Default Behavior/Failure Modes

2007-06-14T10:46:00.001-04:00

(Updated 7/10/07)

Horrific Default Behavior

Today I wrote a batch file called clean.bat and put it in a directory on a different machine. It contained -

del /f /s /q *.obj
del /f /s /q *.ilk
del /f /s /q *.pdb

I had the directory up with Windows Explorer and '\\machine_name\c$\dev\..' was in the address bar. I saved the file into the directory of the other machine. Pretty straightforward, right?

So, thoughtlessly, I double click on clean.bat. Without ado, the batch file starts executing with this message -

'\\machine_name\c$\dev\..'
CMD.EXE was started with the above path as the current directory. UNC paths are not supported. Defaulting to Windows directory.

C:\WINDOWS>del /f /s /q *.obj

Holy crap! Fortunately, I saw it immediately and broke out of it. Even more fortunate, I don't think C:\WINDOWS has any .obj files.

But, if I had wanted to clean .exe files instead (which I frequently do), my machine would have been utterly hosed.

That, my friends, is horrific default behavior.

Failure Modes

Last year, our water heater died. It was pretty old, I think. The problem was the way that it died - by catching fire. That was pretty disturbing.

So the next day, while I was at work, the repairman came out to replace it. Rebekka asked him why it caught fire. The repairman explained, "that's how you know it's broken."Really? The next time my water heater fails, I'll know by following the fire trucks home to see my subdivision in cinders. "Time for a new water heater," I'll say to myself.

And how will I know when my nail clippers need replacing? Oh that's right, from the mushroom cloud over Orlando.

Compiling Haskell

2007-06-06T22:29:00.000-04:00

I've played with the Glasgow Haskell Compiler only a bit. Compiling Haskell to, say, native code is quite a chore. The GHC compiles to a number of special intermediate languages between source and machine code.

Haskell > Core (Hindley-Milner typed lambda calculus) > STG > C/C-- > native machine code

That's a lot of work.

Small Tools

2007-06-06T10:59:00.002-04:00

(Updated 7/10/07)

Part of the purpose of this blog is to remind myself of best practices I've discovered through the years I've been developing software. One of these best practices was brought to mind again in a recent conversation with a colleague.

Small, Combinable Tools

Unix, of course, is built around this philosophy. In essence, it is an extension of the practice of modular programming to applications. It allows us combine the functions of applications in an automatic or semiautomatic way. These are generally command-line tools, which may make them inconvenient in certain circumstances. Nevertheless, with a judicious layering approach, it is frequently possible to capture the best of both worlds - command-line combinable tools underneath, user-friendly GUI on top.

UnxUtils

If you use Windows and you develop software, it is absolutely imperative that you download and use UnxUtils. They are native Win32 ports of many standard unix utilities, and they can be extraordinarily useful. Add them to your PATH. Oh, and if you're not comfortable in unix and, by extension, with these utilities, let me encourage you strongly to learn to use them well. You'll thank me later!

Windows SysInternals

Windows SysInternals. You may not need these beauties often, but when you need them, you need them badly. I believe that few (or none) of these are command-line operable (I may be mistaken).

Windows Utilities

Junction for symbolic directory links in Windows. Get it.
PathMan for path management, and a ton of other good stuff here.
Unlocker, a fabulous (GUI) utility for unlocking files, by Cedric Collumb.

Build Utilities

Ant and NAnt for builds.

Freeing pov2mesh

My pov2mesh utility is currently distributed under the shareware model. I've pretty much decided to release it as freeware. If I do this, I will refund the registered users and remove the registration code from the application. Refunding will be easy, because there are so few registered users! I will not release the full source now, although I may in the future. I've also developed a number of other small tools that I will slowly be polishing (a little, not much) and releasing here sometime.

ASPack

($29) No, it's not a fanny pack. This executable packer is phenomenal.

Not So Small Tools

UltraEdit

($50) My primary text editor. It loads fast and I love its Find/Replace In Files. TextPad's a close second.

LTProf

($50) An easy-to-use, very lightweight profiler.

Araxis Merge

($129) I don't know how I'd live without it.

MilkShape

($35) Basic, inexpensive low-polygon 3d modeler than can import/export almost anything.

Ultimate Unwrap 3D

($50) Unwrap is great for texture mapping.

Paint Shop Pro

($90) Of course, as every game and web-developer who prefers Paint Shop Pro to Adobe's heavyweight (and expensive!) Photoshop knows. Now that Corel owns it, hopefully it will fare as well and not become as bloated (or expensive!) as Photoshop.

Gnu Prolog

Need a good Prolog implementation for expert system development or curiosity or anything else?

In the rare moments when I use Prolog, I use GNU Prolog.

The Impossible Dream #3

2007-05-24T23:11:00.001-04:00

This is, of course, going incredibly slowly and that's just alright.

Script

I'm exploring an interesting architecture. I'm starting to use my script language Pair to script core parts of the game. Pair is Lisp/Scheme-like and very compact (the VM executable [compressed] is 28k without many external functions!). I designed Pair a few years ago with gaming in mind. The original intent was to use it to script many (hundreds or thousands) simultaneous events without having to make everything a state machine. Pair is multithreaded at the virtual machine level and not at the OS level, yielding very lightweight threads. To accomplish this, an explicit frame stack is maintained for function calls, and they are not implemented as C++ function calls underneath. Like Stackless Python which takes a similar approach (and is also used in gaming for its facility in concurrency), it means that Pair can (and does) support continuations because the call stack can be directly manipulated. One of these days I'll put up Pair and source.

On the Side

2007-05-24T21:26:00.002-04:00

In the 80's in front of a house in the Kansas City, Missouri area a man built a boat. It was an incredibly tall half-built steel boat in this guy's yard. Every once in a while we'd drive by and see people working on it and a bit of progress here and there.

One day, my dad decided to stop and talk with the guy, with us little kids in tow. My brother and I took the opportunity to see the beast up close. And it was freaking amazing. To us, it might as well have been a Saturn V.

Many (and many of the best) software developers I've known have had little (or large!) projects on the side. Interviewing developers, I've sometimes asked about their side projects. It's a question that can yield great insight. Where does their passion lie? What problems attract them? What someone does with free time is what they really want to do, regardless of what they might let on in an interview.

It's an indicator of passion and of a drive to create. I've recently come to understand that these side projects also fuel the passion for our day-to-day work and inoculate against burnout. As developers, we have the power to create something from nothing. For some small portion of our time, to follow our own muse can be an exhilarating act.

Some companies, notably Google, have institutionalized this idea, giving developers a certain percent of time during office hours to work on their own projects. I'd expect Google to reap a significant innovative advantage with its policy.

What compels about software development is only partially solving puzzles and, for me, is about the joy and power of creation. Unfortunately, this power is, in part, illusion and the illusion is intensified by software's abstract nature. It nearly always seems far easier to write software than it actually turns out to be.

I'll demonstrate this illusion. Which requires more effort in man-hours: building a bridge across a medium-sized river or developing a game such as Spider-Man 3 or Madden 07? How about the Empire State building or Windows XP? I don't know the answers to these questions, but I don't believe that I can fully trust my intuition to provide them. In any event, good software requires enormous, and univerally underestimated, effort.

Maybe one of these days, I'll have another kooky neighbor, this time toiling away in his garage or barn building a Space Shuttle or something like in that goofy movie "The Astronaut Farmer" (which I have not yet seen). I'll know and he'll know and the fire department and the old ladies from the HOA will all know that it will never lift off. And I'll sit in the garage with that guy, and put the LOX and kerosene tanks together. And it will be freaking amazing.

(Updated 7/11/07)

(Rebekka and I watched The Astronaut Farmer last night and now I wish I hadn't referred to it. I'd hoped for something good, like Apollo 13 or October Sky but Astronaut Farmer was godawful. The premise was utterly absurd and the profoundly self-centered protagonist Charlie Farmer reminded me of an Allie Fox (from a great movie The Mosquito Coast) barely disguised under a veneer of sentimentalism. The best part? The rocket launches from their barn. As in a barn made of wood and filled with hay. And speaking of wood, during one of the bloopers, Billy Bob Thornton asks, in jest, if this was a film directed by Ed Wood. I was wondering the same thing.)

Fun, Fun, Fun

2007-05-05T23:01:00.001-04:00

..until her daddy takes her GTA away?

One of the things that have become even more abundantly clear in the transition to next-generation consoles is how the concept of fun is quite divorced from what we consider a quality game. Quality games must be fun, of course, to a point. They don't have to be too fun, though, and originality in this arena carries an enormous and intolerable financial risk. High-end games must be beautiful to look at, slick, contain vast quantities of content. Frequently, they require online experiences. These types of games must sell millions to recoup development costs. Because risk is so enormous, there is a lot of copycating of successful models.

Let me provide an example. I loved GTA III, liked GTA Vice City, found GTA San Andreas boring and True Crime, Saint's Row and even Crackdown dull, dull, dull. Fun usually requires novelty (or nostalgia), and there's no more novelty left in stealing cars and ramming them into other cars and pedestrians until they explode (the cars and the pedestrians).

(I am a sucker for licensed content, though. I hate to admit it, but it's true.)

Of course, this is something that everyone involved with games knows. But sometimes it's useful to state what's already widely understood.

(Updated 7/10/07)

Grand Theft Auto IV

Ok, I know what I said about GTA-type games getting boring. Still, GTA IV looks awesome. And of course I'll have to play it.

If I get bored by it I guess I'll only have myself to blame..

HDTV's Golden Ratio?

2007-04-18T08:47:00.000-04:00

Is the 16:9 aspect ratio of movies/HDTV (a ratio of 1.777..) more attractive than the 4:3 of Standard Def (1.333..) because it is closer to the golden ratio (1.6180339887..)?

Rebekka says she believes that it's intentional. At first I didn't think so, but now I agree. The golden ratio is a crucial concept in architecture, the visual and musical arts, so it would make sense.

Senzee's Developer Interview Test, Q. 1

2007-04-12T02:50:00.000-04:00

You have two hours to complete this exam.

1.) Without looking at the following questions, quickly estimate how long it will take you to finish this test correctly. Please do not let your estimate exceed the time limit given for this test.

..

..argh..

Moving Parts

2007-04-06T00:44:00.001-04:00

This is where I intend to catalog (all?) the components of a commercial game. I will periodically update this post.

Why?

I've bought and read many books on game development. The bad ones are generally 1000 pages of C++ source code. The good, useful ones tend to fall into two categories.

Narrow and deep (books that equate 'game engines' and 'rendering engines', for example)
Broad and shallow (books that contain light sections on rendering, sound, networking, data structures, etc.)

Still, even books in the broad category aren't that broad. The reason? Books can only be so thick. My point really is that the practice of game development includes a vast array of subspecialties and I'd kinda like to enumerate them here. Hopefully with input from my readers (that's you, Mom) that I'll add. So this list will be ultra-broad with no depth whatsoever. Depth will have to wait for subsequent posts.

Now, not every game will have or need all these systems, but it's crazy just how many most games will need. So, for right now, this is a brain dump and I've only just begun..

Offline

Content Creation - tools - geometry, mesh creation
Content Creation - tools - geometry, mesh creation - offline CSG tools
Content Creation - tools - image creation
Content Creation - tools - motion capture cleaning
Content Creation - tools - higher level 'world' assembly, including assigning gameplay attributes to objects
Content Creation - tools - sound creation/mixing/production
Content Creation - tools - mission/level creation
Content Creation - tools - statistical data, skill/inventory data and attributes
Content Creation - tools - state machine creation
Content Creation - tools - script debuggers, interactive interpreters for testing
Data Management - managing generated code
Data Management - asset dependency management
Data Management - geometry processing - ie. adding attributes, triangle stripification
Data Management - texture processing - texture packing, format conversions, bit depth conversion, etc.
Data Management - asset databases (SQL, XML)
Data Management - metadata collection about assets, data mining for global performance enhancement, data mining for content creation automation, etc.
Data Management - version control systems
Code Management - Code Generation - from asset databases, other offline sources
Code Management - Build System - standard way of building the game for all developers
Code Management - Library/Component Management - system(s) for cataloging, versioning and incorporating libraries and reusable components
Code Management - version control systems
Testing Systems and Methodologies

(much more offline/pipeline side to come; the game pipeline gets far less attention than it deserves, often with disastrous results)

Runtime

System - threading libraries/primitives
System - error handling, logging, assert handling
System - string handling, regular expressions
System - elementary data structures, STL, etc.
System - fast math primitives
I/O - File - standard file i/o
I/O - File - file compression/decompression
I/O - File - loading game assets (meshes, textures, scripts, game data files, etc..)
I/O - File - low-level game asset caching
I/O - File - asset 'packaging' into larger files
I/O - File - object serialization support
I/O - Network - lobby
I/O - Network - server side
I/O - Network - event handling (see Events below)
I/O - Network - publishing player data to online services
I/O - Network - web access services
Localization Support - text databases
Memory - efficient game asset caching
Memory - memory pooling systems
Memory - garbage collection systems
Events - event handling/dispatching systems
Events - 'replayable' game state management
Events - game state represented as loadable/saveable data stream
Events - game state policies - random/deterministic, etc..
Events - game 'flow' management
Sound - Mp3, WAV, Ogg, etc. playback
Sound - runtime mixing, filters
Sound - in game capture
Sound - stereo, 5.1 surround sound
Sound - 3d audio
Input - console controller
Input - force feedback (output)
Input - spatial (Wii, SIXAXIS)
Input - mouse
Input - keyboard
Input - other (Guitar Hero, EyeToy, DDR, Donkey Konga)
Graphics - rendering - geometry processing
Graphics - rendering - geometry processing - progressive meshes and LODs
Graphics - rendering - geometry processing - efficient spatial subdivision
Graphics - rendering - shader(s), per pixel lighting
Graphics - rendering - scene graph management
Graphics - rendering - GPU based utilities for non-graphics
Graphics - rendering - geometry processing - in game CSG operations
Graphics - geometry processing - human animation
Graphics - geometry processing - motion capture
Graphics - rendering - decaling
Graphics - rendering - particle systems
Graphics - rendering - dynamic skyboxes/skydomes
Graphics - images - loading/encoding image formats - DDS, TGA, JPG, etc.
Physics - collision detection
Physics - collision detection - geometry processing
Physics - collision handling, mesh processing for non-rigid object effects
Physics - ragdoll animation
Physics - wind, environmental effects
Gameplay/AI - script
Gameplay/AI - state machine systems
Gameplay/AI - classical AI algorithms
Gameplay/AI - events - collisions, sphere of influence effects, NPC line of sight, NPC earshot
Gameplay - Cars - Damage Models
Gameplay - Cars - Driving Models
Geometry - Modeling
Geometry - Inverse Kinematics
Geometry - Rigging/Tagging/Markup
Geometry - Animation
Geometry - Procedural Animation (ie. Cloth, Hair, Jelly/Spring Models, etc..)
Geometry - Facial Performance and Lip Sync'ing
Rendering - Textures
Rendering - Shaders
Rendering - Animated textures/maps
Video - codec(s)/playback - MPEG, MJPEG, WMV, VP6, Bink, Madcow, etc..
Video - color space conversion
UI - rendering
UI - event handling systems, architecture and data
UI - script
UI - some games have a large amount of UI (ie. Madden, with its enormous UI)
UI - HUDs
Data Management - embedded SQL database (not just for plain-old IT anymore)
Embedded Control - script languages
Control - in-game gameplay tweaking utilities, runtime assets updating support

(more..)

Online Game Services

Online - lobby
Online - server side
Online - (deterministic) player position prediction and tracking (online multiplayer)
Online - publishing player data to online services
Online - web access services
Online - billing tracking for online services (online multiplayer)
Online - player data storage
Online - accessibility from PC website

(more to come, but I'm sleepy now and I want to go to bed..)

Optimal Bytecode Interpretation

2007-04-04T23:50:00.001-04:00

I always feel like I should blog more about engineering related stuff, since that's what I do all day.

For now, my earlier plans for bytecode to C++ compilation are on hold as we're looking into more immediate gains from aggressively optimizing the interpretation. One of the downsides of the hybrid C++ compilation/bytecode interpretation approach that I have in mind is that it will require a workflow change. The C++ will eventually need to be compiled and then linked to the executable. There's a pipeline/workflow bubble there that's not appealing. Still, I think it would be fantastic to get that built for critical core libraries of script (utilities) that don't change much.

Optimizing Interpretation

Our bytecode is that of a dynamic, and (as a result of the structure of the instruction set) terribly slow language. Unfortunately, we use an off-the-shelf compiler, so we can't infer much that was in the original source but fails to show up in the bytecode output.

Did some memoization, in this case caching of variable and function lookups, with excellent (awesome!) results.

A clever universal hashing scheme for both special strings and user strings opened up a lot of possibilities.

Close attention paid to string handling yielded good results.

Thread synchronizaton primitives were killing us. We usually try to keep our assembly language to a bare minimum for portability sake. This translates to a handful of primitives to improve PS2 performance. But to rid ourselves of our most time consuming locks, we applied asm to a couple of critical spots where we need to read in, modify, then write out whole cache lines as atomic operations on the PowerPC machines (PS3, 360). This avoids the expense of a more heavyweight lock such as a mutex in those places and gives us back a lot of performance for multithreaded operation.

Anyhow, we've significantly improved the bytecode execution performance, which is something the game teams have been asking (begging!) for for ages.

Of course, it's never fast enough.. ;)

Language Vector

2007-03-30T22:20:00.016-04:00

"What cannot be spoken of cannot be defeated."

My head isn't clear and I pass in and out of sleep. I'm infected.

Stay with me as I type this. I'll give you a little background. In the 1950's the last of the South Fore engaged in ritual necrophagia and the prion disease Kuru recently claimed the last victims among them. The last that ate the thoughts of another. Nearby, another Papua New Guinea tribe discovered a more pernicious and far more horrifying threat about forty years ago. For years, their population shook and trembled for a reason unknown to them. They pointed to a cave in which words approximated by the English phrase The Black Whisper of Death were inscribed. "The plague came from in there," they said.

8-43 0x12a_rr . ;09]]

Globalization efforts brought the whisper heard round the world. Oh, look! On the overhead TV, that image..

A beloved American religious leader at a political celebration passes an attractive, conservatively dressed young woman. As the light of recognition flares in his eyes, he warmly greets her by name. She smiles mechanically. Suddenly horrified he whirls around to locate the camera capturing an unmistakable expression of humiliation.

It's on TV every other day. She was a minor porn star. It was not that he greeted her that condemned him. He could have legitimately known her in a number of ways. Even had he known her biblically it would have been taken in stride. Who cares? It turns out his televised shame indicted him more decisively than a living-room stash of indecent DVD s. After all, it is shame and little else that condemns us. The tabloids carried that image for months next to the articles decrying Death's Whisper. It doesn't matter. The young woman and the religioso are dead now, victims of their own fascination with their tabloid fame and more directly, victims of its adjacent coverage.

In this case it is knowledge that destroys. dsf6 -saf ;; This is not a disease borne by a bacterium, a virus, or a prion. See, this whisper is a disease carried by the most insidious of vectors - language itself. To simply understand what it is is to be irreversibly afflicted. What cannot be spoken of cannot be defeated. I am condemned already and by writing I damn my readers. But there will be no readers..

Some said it was the Creator, dropping in a kill switch to flip at the turn of a phrase. It's adaptive. They named the disease in the native tongue and it killed them. Scientists described it in English and French and Spanish. It killed them. The blind read of it in Braille, the deaf signed in ASL. Experts from around the world spewed a rainbow of euphemisms. It killed millions. It is not bound to any particular lexical form - perhaps its representation in the brain creates the disease ex nihilo. I don't really know. We did not discover the mechanism that drives this epidemic.

It may be strange to think of language spreading disease, but it is not unique. What is it that viral machines inject into living cells if not RNA/DNA coded assembly language pathogens? I know that machines aren't affected because words are transparent to them. Still, machines played the pivotal role in the spread -&89sd(( of the disease as its immune and efficient carriers.

Can you imagine (no, you can't - because there is no you) seeing all those around you die for an unknown cause of death? As death certificate handlers really understood the meaning of unknown cause of death they came down with the disease in droves. Suddenly, the certificates began listing things like Onychocryptosis and Internecivus raptus. Those who witness the deaths but don't know the cause are inexorably driven to discover it. (Most of) those who know, and are infected, try desperately to not reveal it to anyone and have always failed. Now though, strangely, I'm compelled to type this even if I have to type with the keyboard upside down to keep the blood from my hands from running into it. I can no longer hope for any resolution but the obvious.

And I wait for that resolution even as the asymptomatic machines auto-hyperlink this document. No future, fatal question will go unanswered action diminishes c0ntracts freedom entered invalid through fear judged excuses penalty attached contrary vectpr for reflex arc st1mulus unexpexedly 0kers afwe ~pwnd xasl gatattaccaagggtc link <> link 032 . . .

(story copyright (c) 2007 Paul D. Senzee)

Bytecode to Native Compilation

2007-03-21T18:59:00.001-04:00

At work, some of the software I develop uses a bytecode interpreter. And, as always, we need better performance from the whole system. So I'm looking into bytecode-to-native (in this case C++) compilation. I've done this before, with embedded Lisp-based languages and there are a number of compilers available that do this for Java (GCC has a back-end for this), C#, Lisp and its derivatives (Bigloo for Scheme, for instance). Compiling to C or C++ is great as it serves as a sort of portable assembly language and it's possible to leverage further the fine optimization skillz of modern C/C++ compilers. I'll report on this when I make some progress - if I don't get pulled off onto something else.

Jet Engines

2007-02-28T17:17:00.001-05:00

Creating high-performance technology in a competitive field? Managing your developers' weaknesses and ignoring their strengths may cost you.

Let me illustrate.

Iron accounts for 35% of the Earth's mass. And from iron comes carbon steel (iron and carbon) which accounts for 90% of steel production. As metals go, steel is cheap, abundant and easy to work with. Steel is versatile and good at most everything metals do. If steel was your employee, it would be a damn fine one. You might want all your employees to be just like steel. After all, compared to other materials, it has few weaknesses - but also few extraordinary strengths.

Steel can't do everything. It distorts at extreme temperatures. Steel is not brittle like ceramics but it's also not nearly as hard. It is much cheaper than titanium but weighs almost twice as much. It is a poorer electrical conductor than gold. And of course, steel's strength and hardness in extreme conditions doesn't hold a candle to the single crystal nickel-titanium superalloys* from which they carve jet turbine blades - albeit at enormous cost.

Every one of these other materials demonstrates incredibly high performance in a narrow space and is either prohibitively expensive or ineffective in other arenas.

Even so, it is impossible to build a modern, efficient jet engine with just typical carbon steel.

The bottom line?

Profound strength and profound weakness often come together. Yet without a spectrum of those profound strengths at your disposal, it won't be possible to develop cutting-edge technology at a competitive level.

I don't mean to say that a well-rounded developer can't also have profound strengths - I know several of these people. It's a no-brainer that you need to do what it takes to keep these gems. Reward reliable and well-rounded developers for their lack of weaknesses and, if you need their contributions, reward brittle, unorganized, high-maintenance, socially-inept (insert favorite weakness here) developers for their strengths. Or someone else will.

Notes

* Some of single crystal superalloys are nickel-iron alloys. If iron is the dominant metal in the alloy, these superalloys might be considered steel.

For More Information

Now, Discover Your Strengths

Rolls Royce Trent Aeroengine

Excellent PowerPoint of the Rolls Royce Trent Aeroengine series commercial jet engines
Rolls Royce Trent Aeroengine engine materials

Wikipedia: Iron Steel Ceramic Cermet Silicon Carbide Superalloys Complex Metal Alloy Composites

Development of Single Crystal Superalloys

Senzee's Rule of Interface Complexity

2007-02-06T23:52:00.000-05:00

If what an interface is supposed to do, how it's supposed to do it and the role it plays in a system is not apparent in five minutes of staring at it, it's too complex.

The Impossible Dream #2

2007-02-06T14:54:00.001-05:00

Previous: The Impossible Dream #1

Ground Rules

Mantra 1: The mechanic first, then the game.

It's no fun putting blood, sweat and tears into an unfun game. We structure the game around a solid mechanic - prototype first to find addictive key element(s) and elaborate. Tag is one key element we've identified.

Mantra 2: Build a game, not an engine.

I've seen a number of home and indie game projects go south stuck at the "let's build our engine" phase. Nothing overly general - we build into the code what we need to do the trick. I'm not saying there's no place for useful abstraction here, I'm saying that a general purpose rendering or game engine is not our goal.

Mantra 3: Fit the design to resources at hand.

Building an open world game at all is our first violation of this. Still, we need to embrace this reality instead of fighting it. Operating with restricted resources can force us to come up with creative solutions to problems we otherwise wouldn't have looked at. Of course, it may also prevent us from finishing anything ever.

Mantra 4: Don't underengineer or overengineer, but if you must choose one, underengineer.

Why? Because underengineering is easier to fix.

A Unflinching View

Let's start the impossible dream by looking, somewhat realistically, at our available resources and the strengths and weaknesses of our position.

Resources

A few development hours a month divided among game design, software development and content creation.
A powerful target platform (PC).
Development tools - Visual C++ 2003.
Direct3D and its auxiliary libraries (see No need for portability in Strengths, below) .
Third party content creation tools - Milkshape 3D (registered), Unwrap3D (registered), POV-Ray.
Open source and free resources on the web for game development.
Free content from other sources that can be legally used in our game.

Strengths

Strong software development skills.
A powerful target platform (PC).
No need for portability.
A great deal of freely available software for offline and runtime use.

Weaknesses

Poor content creation ability.
Few hours to dedicate to the project - it'll be done in like one hundred years.
Little monetary investment.

Strategies

Make the most of our particular set of resources:

Simplicity - a simple, compelling core game. No minigames, nothing like that.
Prototype the game mechanic first.
Automation - machine generated content, mostly offline, possibly at runtime.
Reuse/scavenge (to a point) freely available content generation tools and software.
Reuse/scavenge (to a point) freely available software for the runtime.
Seek out free content that we can legally use in game.
Creatively adapt the game design, including character and plot, to cater to our strengths.
Use a hooded figure for the main character, allowing us to skirt the animation issue for the main character. We can use a simple segmented model for NPC's. We can animate the hood & clothing of the main character with cloth animation, which is CPU-intensive, but nearly content-free.
Use public domain (classical, folk music?) music for the soundtrack.

Oh, and yes, there's so much more to come.

Red 5's Pitch

2007-02-05T20:41:00.004-05:00

(First off, special thanks go to Ryan Burkett who allowed me to use his digital camera to photograph these items.)

Last week I received a Fed-Ex package at my desk. All excited to get a package, (kind of like those people who win on the radio and say "I never win anything!!") I grabbed Ryan Burkett to open it with me. Inside there was a ~3"x~6"x~1" paper box that looked like some sort of puzzle. The box was orange and black with an artist's rendering of the birth of a planet. It carried a large white digit 1 in the lower left-hand corner. Small print above the 1 read:

R5L_001 TRACE
not the end
a beginning
a new beginning

One corner was cut out and a slightly smaller box was visible inside. I opened the first and pulled out the second. It was similar, with olive green instead of orange and a large 2 in the corner. It sported an illustration of a shard of molten rock (or perhaps the broken wing of a spacecraft?) and read:

R5L_002 TRACE
not a new world
but an old world
made new

So, I figured this is some kind of Matryoshka doll package. I pulled out the third box, sea-green, marked with a large white 3 and a primordial underwater scene.

R5L_003 TRACE
no change is peaceful
and through life struggles
it also thrives

Out came number four, peach-orange with a river of lava flowing through the gaping maw of a mountain pass.

R5L_004 TRACE
to forge a new path
through the darkness
to rise to the call of glory

And then finally number five, red with the image of a blue elf from Red 5's website and the text:

R5L_005 TRACE
not the end
a new beginning
with new eyes to greet it

The fifth opens like a book or a box of software and inside was an iPod Shuffle with my name engraved on it!

Now pausing for a moment, it occurred to me that everything about this is loaded with references that I don't think I fully understand. The images and text seem sort of Genesis with the days of creation and all, but that's six days, not five. So I'm not sure if that's the reference, but it seems an apt associaton with the creation of a new studio, a new game and the new changes of the people's lives who are coming to join Red 5.

So I power up the iPod and listen to track 1.

Paul, this is Mark Kern, President of Red 5 Studios and former team lead for World of Warcraft. I came across your blog on the net and was impressed with the depth of your inquisitiveness regarding game design, programming and mathematics. Your work at Tiburon is impressive and we'd love the chance to meet you. At Red 5 we're assembling a team of incredibly talented individuals dedicated to pushing the envelope in online entertainment. We're building a new type of game company and a new type of game. And we believe you're someone who just might fit into the Red 5 family. Log into red5studios.com and enter the code found engraved on this iPod. We'll tell you all about it.

So I pull up the site:

Who are we?

Red 5 is a creator-owned, newly formed studio driven to create new and orginal massively multiplayer online entertainment. We are world-builders, storytellers and game makers. We own what we make, and we have the financial backing to stand on our own.

So Here’s Why We’re Calling.

Red 5's CEO came across your blog and was impressed with the depth of your inquisitiveness regarding game design, programming, and mathematics. Your work at Tiburon is impressive, and we'd love the chance to meet you.

Because of this, you are one of one-hundred people in the industry we've invited to check out Red 5 Studios. We've played your games, and respect your skills.

Paul, we are hiring, so feel free to take a look at our open positions. We had a few in mind that might be a good fit, but we really believe in people and, specifically, your potential. If you don't see something that fits, we can talk about tailoring something to your specific talents.

Pretty impressive pitch, huh? Certainly very flattering!

Here's the original post.

The Impossible Dream #1

2007-02-02T17:26:00.002-05:00

True Story

In Mexico I met a poor farmer named Miguelito who lived in the little town of Metepec (Atlixco), Puebla. Many years before, in a vision, the Holy Virgin of Guadalupe entrusted him to build a cathedral, high up on the hill above the town. She promised him that he would not die until he'd set the final stone. He quickly enlisted the help of the townspeople and over 15 years constructed a small, beautiful hilltop church. At the time Miguelito showed me the rust-colored church it was nearly complete and he explained that this meant that his time was near. His mistress had been generous enough - Miguel was 104 years old!

My Impossible Dream - My Own Open World Game

So, is it possible, working a few scant hours each week on evenings and weekends to build a (simple) open-world game by myself? We shall see. I've occasionally been working on the game since about a year before I came to EA but I think I'll make a more public spectacle out of it on this blog - to keep me motivated.

So why do I even want to do this? Because it's impossible.

Some prototype screenshots
Some generated content and concepts

Next Time: in the next installment of The Impossible Dream, we'll look at what this game will be about and how we will use automatic content generation to build our world.

Red 5 Studios

2007-02-01T16:50:00.001-05:00

This week I received a package from Mark Kern, former team lead of World of Warcraft and current CEO of Red 5 Studios who is, it turns out, a fan of my blog! Before I delve into more, let me say that Mark and company have the coolest recruitment strategy I've ever seen. And although I'm not looking to leave EA Tiburon at this point in my career, perhaps there's a future out there..

Mark and I spoke for some time about his company. Red 5 aims for the center of the emerging MMO phenomenon in ways that transcend east-west cultural boundaries to create an experience as compelling for Chinese and Korean gaming communities as it is for American, European and Japanese communities. What's more, Red 5 is staffed with a large number of ex-Blizzard game makers who know how to build the kinds of MMOGs that can do this. As an American company at ease with Asian cultures, Red 5 has given an enormous amount of thought to handling the cultural nuances to make this work. I'm not an expert when it comes to MMOs, but I'm convinced. If there's a developer to back in the MMO space, it's these guys.

Red 5 Taps Some Green
Red 5 Studios Aims To Be Pixar of Online Gaming

7

2007-01-24T20:08:00.002-05:00

(Updated 7/10/07)

Inspired by the interest in my 5-card poker hand code that plugs into Cactus Kev's evaluator, I've decided to revisit my unholy 7-card evaluator and make a faster?, cleaner one that I can then post up here.

For the 5-card hash I used Bob Jenkin's Perfect Hashing code. Check out his excellent site for great perfect hashing code & ideas.

My current 7 card evaluator first determines if there is a flush. If not, it looks up the final value in a 13 * 13 * 13 * 13 * 13 * 13 * 13 (13^7, 63M entries) precalc'd table. Arghh! If it is a flush, though, it evaluates all 21 combinations (7 choose 5) in the normal (albeit optimized) way.

But this is not how I want my grandchildren to remember my code. Let's think about other options. Now { 52 choose 7 } yields about 133 million possiblities, right? The first crucial step in thinking about optimizing the seven hand evaluation is figuring out a way to efficiently map every unique set of 7 out of 52 cards to one unique number of the 133 million possiblilities.

As it turns out, I've got some code to do that. Nevertheless, I need to do a little cleanup before I post that. So look for "7 Part II" sometime soon.. ;)

Part II: 52 Choose 7

As promised, code to map any 7 of 52 items (7 of 52 bits) to a unique index in the range of 0-133M (52 choose 7).

index52c7.h

This, of course, could be used for a super-fast 7 card hand evaluator with a precomputed table of size 266mb.

Jing, commenting below mentions that a 2+2 forum has some super-fast seven hand evaluators. Glancing briefly at the site I notice claims of 12.5 cycles per evaluation, which seems too good to be true. After all, a single out-of-cache table lookup can cost much more than that. But if it is true - sweet!

Some Clarifications

Andy Reagan emailed me and made some excellent points concerning the readability of index52c7.h.

"[It's hard] to understand what the code was doing without comments and with the generalized table and variable names.."

I apologize for that. Actually, I wrote another program to generate this file, which is one of the reasons why it's so obtuse. It would probably be a good idea to publish the generator program as well.

"What does the function index52c7 do?"

Here's the reasoning for index52c7:

We can completely represent a hand of 7 cards of 52 with a single 52-bit number with 7 bits set. We assign each possible card in a deck a number between 1 and 52, inclusive. For example, the Queen/Hearts might be 43 and 2/Spades might be 17. Then, we take a 64-bit number (large enough to contain the 52-bits) and set a bit for each of the 7 cards we have. If two of the seven cards we have are Queen/Hearts and 2/Spades we'd set bits 43 and 17 along with the bits that correspond to the other five cards.

Now, if we had unlimited memory, we could just use this number as an index into an enormous and very sparse array. Unfortunately, this array would have 2^52 (4.5 quadrillion) entries. Assuming two bytes per entry, that would require 9 petabytes of memory! So we need to somehow hash this number into a much smaller space. It turns out that the number of possible combinations of 7 items among 52 is about 133 million (52 choose 7), so ideally, we could somehow hash the 52 bit number into a number between 0 and 133 million that uniquely identifies a given hand.

That's what index52c7 does. It translates the 52 bit hand representation into a much smaller, but still unique number. At two bytes per entry, that gives us a table of 266 megabytes, which is large and in certain cases inconvenient, but certainly doable.

Using, say, Cactus Kev's code to evaluate each possible 7-card hand, we'd first generate the 266MB table and populate it by looking up the corresponding index with index52c7. Now that the table's fully populated, we can just pass index52c7 the 52-bit number and use the resulting index to pull the answer straight out of the array.

Floating Point

2006-12-08T00:15:00.000-05:00

An excellent article on working with floating point numbers.

Explorers and the Game of Tag

2006-12-05T21:29:00.001-05:00

I'm partial to games of exploration. The Ultimas and their relatives were my early favorites. These days, I play games like Grand Theft Auto, Godfather, Spider-Man 2 and Oblivion.

Why? It's the joy of discovery - the pleasure of finding things out. So what makes a great open world game? In the context of video games, an open world is a toy upon which a game can be built. So first, what makes a great open world?

Big, not too big

It's no secret that we measure the size of our universe in the time it takes to cross it. The world shrinks with the inventions of the railroad, the automobile and the airplane. Unlike other open world games, the world of Superman Returns seems in turns enormous and tiny, depending on whether you are walking or flying at supersonic speeds. When a world is too big, we can spend all of our time simply getting around. Too small and it seems cramped and unimaginitive.

There must be some novelty to every distinct area or region. The 1987 game The Faery Tale sported a world of over 17,000 screens. There were so many repeating patterns of landscape and it took so long to cross that it was hard to stay interested in exploring the world. In an interview with Game Informer, Radical, in creating Hulk: Ultimate Destruction, specifically addressed this issue of variety.

Mechanics

The mechanics of a world are crucial to its experience. They are the laws of the game's universe. It is important that the game itself cannot subvert these rules. In GTA, this includes the way cars move collide and interact, that any car can be rigged with bombs and the general behavior of the people and objects that inhabit Liberty City. Another important mechanic in GTA is police response to player actions. The world of GTA is rich with mechanics largely orthogonal in purpose. It is this richness that makes GTA great. In Spider-Man 2, the central mechanic is web-swinging. The world of Spider-Man 2 is designed to be experienced through web-swinging and so it is an inseparable element of Treyarch's Manhattan. In GTA, Spider-Man 2 and Superman Returns the speed at which one is able to move through the city is variable and its 8control is in some mechanic anchored in the game world itself. This provides a sense of enormous freedom. In some of these games, this speed increases as the player progresses through the game.

Even so, at this level, the open world is still a toy. Now let's ask the question, what makes a great open world game?

The Childhood Game of Tag

Of course, GTA 3 is the prototypical open world game. Recently, I started playing it yet again and it still may be the best. What makes it fun? The underlying element is tag. The ancient children's game is incredibly simple, infinitely flexible and has strategic depth. Tag benefits from richness and complexity in a large world. The ideal tag world has nooks and crannies in which to hide from predators and from which to ambush prey. Layered on top of that, in GTA, there a number of other game mechanics, many - most? - of which are themselves variations on tag.

These layers of tag and alternating roles of hunter and hunted in GTA 3 stand in contrast to Spider-Man 2's races which also aim to capitalize on a large in-game world. Unfortunately, racing is much less fun than tag. Even though it's one of my favorite games, Spider-Man's races don't appeal to me. What appeals to me about Spider-Man 2 is the absolute euphoria of swinging and the crazy missions (levels?) like Mysterio's burning theater and the Statue of Liberty.

Tag (from Wikipedia)

Players: 2+
Rules complexity: Low
Strategy depth: High
Random chance: Low
Skills required: Running, Hiding, Observation

UML's Accidental Complexity

2006-12-05T08:48:00.002-05:00

Accidental complexity is the complexity imposed by a given approach (in this case, a language) to solving a problem as opposed to an essential complexity of the problem itself. Recently, I documented a system I'd developed for a presentation for my coworkers. I diagrammed the class relationships in UML in Visio. Then I started swearing as I always do whenever I use UML. Now don't get me wrong, I enjoy UML and it's a valuable and useful tool when its limitations are understood.

So why does UML make me swear when I use it?

UML is graphical and two-dimensional.

A UML design should be readable, preferrably at a glance without too much study.

UML is general, intended to apply with equal descriptiveness to multiple object-oriented languages.

These goals of a UML design are irrelevant to the original language implementation. And, of course, the implementation language will have its own accidental complexity whose concerns must take higher precedence. Nevertheless, readability is important. An unreadable UML diagram is a waste of everyone's time.

What effect does UML representation have in describing software designs? Since subclassing allows classes to inherit lines of association from their parents while containment does not, readable UML favors subclassing over containment/composition which, in C++ based object oriented languages (such as C++, Java and C#), is inappropriate and even wrong (later, an entry on subclassing vs. containment).

Below, an artificial abstract class is introduced to tame UML complexity. Keep in mind that this diagram represents a very simple design.

Before -

After -

The first diagram resembles an electrical circuit and its suffers from the same difficult problem - complex, multidimensional relationships flattened to a two-dimensional plane. In the second, we introduce a false abstract class simply to preserve readability.

Clearly, readable UML representation as a requirement for software design is not a useful constraint - especially when it degrades the quality of a software system. This is an issue that must be understood when using UML to design or present software systems.

On Reference Counting

2006-12-04T21:49:00.001-05:00

Reference counting is one of those nasty little things that always seems easy and straightforward before you do it and for which you always curse yourself afterward. There is often little choice (so don't curse yourself too much) because in many circumstances it's the lesser of two evils. The other evil is full blown garbage collection - arghh..! Remember, if there's a good way to restructure your code to eliminate the need for ref-counting or garbage collection, DO IT!

I have code that I use whenever I decide to ref-count that allows me to monitor object lifetimes, object ages and so forth to easily debug ref-counting issues. It requires extra memory and so is not suitable in every case. Alternatively, I've simply logged object creation and deletion to a file and then written a separate program to read the log to locate the offending objects and the general regions of code responsible for creating and (not) deleting them.

But don't ever assume that ref-counting bugs will be easily fixed. Usually once you fix one symptom another one pops up somewhere completely removed from the first. Both are probably unrelated to the source of the problem. This is what makes ref-counting insidious. So take my advice, when implementing ref-counting, do your due diligence upfront and make sure every increment and decrement is accounted for. You'll be sorry if you don't.

Inline Assembly

2006-12-04T20:57:00.001-05:00

On my current project I will soon delve into optimization tasks at the level of inline assembly for PowerPC. These days the use of inline assembly is almost never justified. It's about as unportable as code can be and it's nearly impossible to understand once it's written. Most of the time, unless you devote a great deal of energy or unless you are using processor features (SIMD, for example) inaccessible through C or C++, hand-written assembly will actually be slower than compiler generated code. Furthermore, most of what you learned a few years ago about optimizing assembly code simply does not hold any more. For example, what's the faster way to multiply an integer by 5 on the x86?

A. x = (x << 2) + x;

or

B. x *= 5;

Old school assembly says that A is faster. Not true anymore. The imul (integer multiply) instruction is as fast as a single shift on the x86 these days. Counting cycles? Hard to do these days with deep pipelines, instruction reordering, branch prediction and unpredictable memory latency. The most effective way to optimize assembly seems to be aggressive profiling and trial and error. Gone are the days when you can optimize code by counting cycles with the processor manual tables in hand. Even so, these guidelines are important:

1. Most importantly, make sure you have the most efficient algorithm possible for the job before moving to assembly! There are a million good reasons for this and nothing could be more embarrassing than having your finely tuned assembly bubble sort owned by a C (or Java!) mergesort written in 12 minutes.

2. Profile changes aggressively and with the finest resolution (usually the CPU cycle counters) possible.

3. Space out memory accesses. Because of memory latency (and asynchronous memory access), you can hide cycles between your memory reads and writes.

4. Know your memory access patterns and take advantage of them. Do you only write and never read back from certain areas of memory? It may be beneficial to write-through directly to memory and avoid caching. It can also be useful to prefetch memory in certain cases.

5. Keep your data structures small enough to fit completely in cache. This will yield enormous benefits if you can do it.

6. Use SIMD where appropriate. This can give great benefit and itself may justify moving to inline assembly. However, don't spend an excessive number of cycles trying to fit data into SIMD-ready structures. It'll probably cost more than you'll get from it. Use SIMD when it's a good fit.

7. Unroll loops - to a point. Unroll tiny loops until they no longer provide a performance benefit. Keep unrolling and profiling. When you've gone too far you'll see a significant performance drop as that piece of code outgrows the instruction cache. If you have enough information on the hardware, you can figure out where this threshold will be.

8. On PC use SIMD for 64-bit integer arithmetic instead of the atrocious code that's generated for this by Visual C++.

Just so you know, this entry is subject to revision. Have any other guidelines? Let me know about 'em!

Experimental Gameplay

2006-12-02T11:59:00.000-05:00

I met Kyle Gray yesterday who sits a couple of rows away from me at work and runs a fascinating site called the Experimental Gameplay Project. If you are at all interested in game play, check it out.

Some of my favorites so far are Hovercrafty, Pluto Strikes Back and Big Vine. Maybe around Christmas when I've got a little time off, I'll make a game or two.

Some People I've Met (and Some People I Haven't) Whose Work I Like

2006-11-28T17:17:00.004-05:00

Earlier I mentioned Richard Garriott as someone I'd like to meet. Jamie Fristrom emailed in response to that and sent this interesting story.

In junior high, I discovered Hackers, by Steven Levy which I must have read a dozen times. Steve Wozniak became my hero. I wanted to build computers.

Then it was games. I wrote to all of the programmers who converted Times of Lore to PC asking how they accomplished the overhead scrolling over the massive world. Herman Miller responded with a detailed description in C and assembly and recommended that I get the Programmer's Guide to PC and PS/2 Video Systems which I guess is going for $0.24 on Amazon these days. Armed with that information, the book and Turbo C, I started writing games like crazy.

So in college, a friend, Alex Kapadia, and I conceived a game called Undead: The Tibovian Era. I sent an early copy to the fast-growing shareware company Epic MegaGames and a few days later, received an excited phone call from the now legendary (in game circles) Tim Sweeney. Tim sent me an email (which in those days I dutifully checked on the Vaxen at UMKC) saying, "I'm looking forward to seeing more of Undead! It's the best 'first' game any author has shown us." Not bad, eh? This was a year or two before Unreal and I like to believe that my Undead inspired the similar name of Tim's magnum opus. Who knows..

Instead of finishing Undead, I spent a few years in Mexico. Kids these days. When I returned to Kansas City I found employment for a while doing bilingual customer service at a well known electronics company. At that time, I found myself sitting across the aisle from Niki Sullivan, one of the original members of Buddy Holly's Crickets! In his honor, the call center played nothing but Buddy Holly. Like an idiot, in my months there I was too awestruck to ever ask him anything about the days he wandered around the country participating in some of the seminal moments of American pop music.

And, of course, I've met a number of equally talented people in places I've worked such as Keane, VML, H&R Block, Sprint and EA Tiburon.

Without further reflection, that's the story of some people I've met (and some people I haven't) whose work I like.

Speech Impediment

2006-09-13T21:15:00.001-04:00

(Updated 7/10/07)

Lisp (and to a lesser degree, Scheme) fascinates me. Simple in the extreme and yet incredibly powerful, it is a thing of beauty. I learned Lisp by writing Lisp interpreters and compilers - usually to embed in other applications. As an embedded language Lisp excels - it is among the easiest languages to implement. Implementing it well, however, is an altogether different proposition.

By programming language standards, Lisp (1958) is an ancient tongue. It is older than COBOL by a year and older than C by more than a decade. And it is timeless.

Don't laugh but it's the coolest thing to build a tiny and reasonably fast Lisp bytecode VM (Pair, a current project of mine that I never get to, weighs in at a mere 28K!) and then shove it inside some application and make it do all sorts of things machines were never supposed to. Like thinking, for example :).

Lisp & Scheme

muSE - Embeddable Scheme dialect and more importantly, its associated blog.
Lambda The Ultimate Programming Languages Weblog
Elk

Some Favorite Lisp and Scheme Books

Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp
Lisp in Small Pieces - Great for building Lisp interpreters and compilers.
On Lisp
Lisp
Structure and Interpretation of Computer Programs - Don't buy this classic though - it's free online.

Let Me Count The Ways (Part II)

2006-08-22T20:03:00.002-04:00

In the Sieve of Eratosthenes post, I mentioned that at work (EA) we have an optional programming challenge every once in a while. It was started last year by Jim Hejl and it's called Hacker's Delight - named after the excellent book of programming tricks by Henry S. Warren.

I love this stuff. Why? Who knows? Anyhow, this time I wrote the challenge - Hacker's Delight #6. I chose the { 64 choose 4 } problem from Let Me Count The Ways. My fast solution was initially ~3.9ms. Since I had not yet aggressively optimized it, I did that and got it down to ~2.5ms. That solution was written completely in C++. I figured that was about as fast as it got because it turned out that ~2.5ms is (slightly) faster than memset(data, 0, 635376 * sizeof(unsigned __int64)) !!

In short order, solutions came in that were almost exactly as fast as mine with quite different algorithms. Hmm. I thought, the bottleneck is memory-processor bandwidth - ~2.5ms is simply how long it takes to write 4.8mb of data. All the other processing is swamped by that.

Even so, Jim was experimenting with SIMD approaches and had an SSE version of 'memset' that executed in about half memset's time. I started playing with that but couldn't figure out how to use that to get better performance out of my algorithm.

Then Jim tells me that he received an entry from EA United Kingdom that executes in ~1.25ms - all SIMD (MMX)! The gauntlet is down. I asked him not to forward it to me or let me see it.

So today I finally got my own MMX version to about ~1.25ms. It seems to execute slightly faster than Jim's SSE memset. While minor speed improvements may be possible (on the order of tenths of milliseconds), I'm convinced that there's no way to get significantly faster performance.

I could be wrong.

When it's all over, I will post a more detailed description of the optimization steps for those of you who are interested. For those who are not, z z z z z z z z z z...

[The related, and more detailed document Optimizing 64 Choose 4 (.pdf)]

Some Perfect Hash

2006-06-08T15:50:00.002-04:00

The computer science department of the University of Alberta in Edmonton researches Artificial Intelligence (AI) for Poker. From their site I came upon Cactus Kev's Poker Hand Evaluator which is a killer fast five card hand evaluator. Reading through his algorithm you'll notice that the last step for any yet unclassified hand is a binary search through a list of values. Most hands end up in that search which is the most time-consuming part of the algorithm.

An Optimization

Replacing the binary search with a precomputed perfect hash for the 4888 values in the list yields a significant improvement over the original. The original test code included with Kevin's source (using eval_5hand) runs in 172 milliseconds on my machine and with my eval_5hand_fast it runs in 63 milliseconds. Yay, an improvement of 2.7 times!

fast_eval.c

(Updated 7/10/07)

This post has garnered a bit of attention. Cactus Kev added a comment and a link on his site back to the post and I've had an email conversation with a programmer (anonymous unless he's cool if I mention him) who ported the code to C# and reports:

Kev: 159ms
Your mod: 66ms
My C# version of your mod: 88ms

Pointers for seven hand evaluation? Check out the post 7.

Sieve of Eratosthenes

2006-06-08T11:12:00.002-04:00

At work we had a little challenge in December to see who could come up with a program to count the number of primes between any two numbers (0 to 2^32-1 inclusive) as fast as possible. To this end I wrote this optimized Sieve of Eratosthenes (algorithm) that counts all the primes from 0 to 4,294,967,295 in about 13.75 seconds on my (admittedly fast) development machine at work.

If you come up with any improvements to it, let me know!

(2/2/2007 - I'm going to go ahead and add the link for the multicore Sieve of Eratosthenes here. On a 3GHz dual core Xeon, 7.15s!)

Let Me Count The Ways

2006-06-02T16:44:00.001-04:00

Lately, I've been tinkering with card games (poker) a bit and one of the little questions that came up was what is the fastest way to enumerate all possible combinations of 4 items out of a possible 64? Actually, it doesn't have to be 4 of 64, it could be 7 of 48, or 2 specific aces from 52 cards or 3 bits of 10. The field of combinatorics can tell us how many there are. This is expressed as { n choose r } and has the formula factorial(n) / (factorial(r) * factorial(n - r)). In the case of n = 64 and r = 4 it yields 635,376 different combinations.

So, the task is to enumerate each of these unique combinations exactly one time until all 635,376 have been generated. This is one of those problems where the right approach makes all the difference.

The Naive Approach

Let's look, for a moment, at the most basic brute force approach: iterate through every value that can be contained in 64 bits and check if it has four bits. How long would this take? On my machine it takes 1 second to check 250 million numbers. Pretty fast, right? At this rate, however, it will take 2,340 years to check all 2^64 numbers! Clearly not a usable solution.

A Better Approach

Poking around the internet I found a function that takes a number and returns the next highest number with the same number of bits. I adapt it a bit and bam!


int enumerate_combinations(int n, int r, unsigned __int64 *v)
{
    unsigned __int64 y, r, x;
    unsigned         count = (unsigned)math::choose(n, r);
    v[0] = ((unsigned __int64)1 << r) - 1;
    for (unsigned i = 1; i < count; i++)
    {
        x    = v[i - 1];
        y    = x & -(__int64)x;
        r    = x + y;
        v[i] = r | (((x ^ r) >> 2) / y);
    }
    return count;
}

This variation turns in 32 milliseconds for complete enumeration. Not a bad improvement over 2,340 years, eh?

The Best Approach

I actually developed the following function before the above. However, I figured (from looking at the code) that the above would blow this one away. How wrong I was. One problem with the above approach is that it has a nasty little division. Another thing is that this approach takes advantage of certain special cases like when r == 1 and n == r. The following approach is based on my initial recursive approach, but I removed the recursion so that I could rewrite it as an iterator. Removing the recursion did not seem to have a significant impact on performance. Anyhow the following runs to completion in just 3 milliseconds, over 10 times faster than the above version.


template <typename T = unsigned __int64>
int enumerate_combinations(int n, int r, T *v)
{
    struct { int n, r; T h; } s[sizeof(T) * 8] = { { n, r, 0 } }, q;
    int si  = 1, i = 0;
    T   one = 1;

    while (si)
    {
        q = s[--si];

    tail:

        if (q.r != 0)
        {
            one = 1;
            if (q.r == 1)
            {
                for (int j = 0; j < q.n; j++)
                {
                    v[i++] = q.h | one;
                    one <<= 1;
                }
            }
            else if (q.r == q.n)
            {
                v[i++] = q.h | (one << q.n) - 1;
            }
            else
            {
                --q.n; s[si++] = q; q.r--;
                q.h |= one << q.n;
                goto tail;
            }
        }
    }

    return i;
}

The Elder Scrolls IV: Oblivion

2006-05-30T10:33:00.001-04:00

This weekend I sprung for an Xbox 360. Finally. Then I rented The Elder Scrolls IV: Oblivion. I'll have to take it back to Blockbuster Thursday, so I'm gonna have to buy it. $60, damn. Graphically it is, without a doubt, the most beautiful game I have ever seen. Its gameplay is utterly consuming. I may not be sleeping much these next few weeks.

On coming to the surface (the game starts underground), an incredible panorama opens. The trees, the normal/gloss mapping of the rocks, the water. Everything looks amazing. The world is massive beyond belief. A profound sense of freedom - you can do anything. Playing Oblivion for the first time was very much like opening up the exquisite world of Ultima V as a kid. Few games appeal to me that much.

For me, it is essential that a game inspire a sense of freedom with Ultima V, Times of Lore, GTA III, Spider-Man 2 being representative examples.

Data Access Latency

2006-05-23T09:11:00.001-04:00

Ryan and I have talked about making a chart to easily visualize the relative costs of common computer operations. A significant part of this is data access latencies in modern computer hardware. Yesterday I stumbled across some numbers in Raymond Chen's PDC 05 Talk: Five Things Every Windows Programmer Should Know, and behold! a chart is born:

Main memory latency has become a crucial consideration in modern software development. As the processor (CPU in the chart) becomes faster, the gulf between the processor memory (registers and caches) and main memory latency widens, making cache misses increasingly more expensive with respect to processor cycles. An increasingly frequent software design decision is to use simple array based data structures as opposed to pointer based tree and list structures for many operations. Typically, pointer structures (linked lists, binary trees, etc.) are theoretically more efficient than their linear counterparts (for example, binary trees [O(lg n) vs. O(n)] vs. linear search). Given the high cost of cache misses in modern hardware and the good memory locality of array based approaches, pointer based structures may perform 25-100 times more poorly than their simpler counterparts. In developing for the Xbox 360 and PlayStation 3 with crazy powerful processors, linearizing data structures is a crucial optimization.

So instead of using that binary tree next time, consider using a sorted linear array with a binary search.

class vs. struct

2006-05-18T15:33:00.001-04:00

As C++ interviewees know well, the only difference between class and struct in C++ is that class defaults to an access mode of private and struct defaults to public. This means that the difference between them is purely syntactic and has no semantic connotation whatsoever. Because of this, some C++ experts believe that the struct keyword should not be used at all and we should always use class { public: instead.
So why do developers continue to use both when there is no semantic difference? To people, struct and class communicate subtly different ideas. Developers often use the struct keyword (because of its C heritage) to indicate a lightweight, open record that is not encapsulated. For example, a small record intended to be written directly to a file is more likely to be a struct in these situations. The class keyword is then used for traditional C++ object orientation. The fascinating thing about this dichotomy is that even computer language keywords develop nuances of meaning apart from their original intent.

Formatting std::string

2006-05-18T08:38:00.001-04:00

The following snippet has been a part of my personal code library for years. It is useful for formatting a std::string in a traditional printf() way. For all its ills, printf/sprintf() is incredibly convenient. This code is for Win32. Minor modification is required for Unix.


#include <stdio.h>
#include <stdarg.h>
#include <string>

std::string format_arg_list(const char *fmt, va_list args)
{
    if (!fmt) return "";
    int   result = -1, length = 256;
    char *buffer = 0;
    while (result == -1)
    {
        if (buffer) delete [] buffer;
        buffer = new char [length + 1];
        memset(buffer, 0, length + 1);
        result = _vsnprintf(buffer, length, fmt, args);
        length *= 2;
    }
    std::string s(buffer);
    delete [] buffer;
    return s;
}

std::string format(const char *fmt, ...)
{
    va_list args;
    va_start(args, fmt);
    std::string s = format_arg_list(fmt, args);
    va_end(args);
    return s;
}

Muddy Waters

2006-05-15T17:32:00.001-04:00

A fascinating view of anti-pattern/pattern Big Ball of Mud and its relatives. Big Ball of Mud is the kind of horrifically structured software we have all seen and shunned throughout our careers. Software of this sort is not just poorly architected, but lacks any architecture whatsoever. Nevertheless, the authors argue, when an approach is so pervasive and universal as this one is, there must be something it does well.

Citing shanty towns as an instance of the Big Ball of Mud pattern, the authors say:

Shantytowns emerge where there is a need for housing, a surplus of unskilled labor, and a dearth of capital investment. Shantytowns fulfill an immediate, local need for housing by bringing available resources to bear on the problem. Loftier architectural goals are a luxury that has to wait.

Traditionally good architecture in software dramatically reduces maintenance requirements and failure rates. Values that traditional software architecture respect are robustness, maintainability/low cost of maintenance, performance and efficiency. To achieve these goals, the development of such software requires software architects (not cheap), more sophisticated (and therefore more expensive) developers, and, in general, more development time.

In contrast the Big Ball of Mud development style values low cost of initial development and is willing to achieve that by accepting a relatively high cost of continuing maintenance as its solutions are not robust. It requires no architects, far less skilled developers and less initial development time. However, it may also require a larger dedicated maintenance staff.

Urbanization

2006-05-15T12:47:00.002-04:00

Because I enjoy creating games on my own time and am attracted to the sort of open world games described in the previous post, I'm interested in automatic generation of urban landscapes for games. There is no way I can create an entire city such as the cities of GTA or Ultimate Spider-Man without a high level of automation. I've seen some work in this area, most of which is concerned with generating low-detail cities in real-time. I'm interested in offline high-detail content creation.

From 2004 to early 2005 I developed a pair of programs to accomplish this. The first of these is pov2mesh, a program that converts a constructive solid geometry (CSG) file (Pov-Ray format) into a Maya vector .obj file. The second of these is Urbia which generates the CSG file from an abstract .xml definition of the world, fills in the blanks and invokes pov2mesh to complete the conversion to .obj. The pov2mesh application is currently offered as shareware on my site. I also plan to offer Urbia as shareware when it's more polished.

See this for a discussion of pov2mesh's future and feasiblity.

A city block generated by Urbia for my game in progress Cuervo.

(Updated 7/10/07)

Here are some great links passed on from an EA guy who went to SIGGRAPH 06-

Procedural Modeling of Urban Environments

Peter Wonka's Publications

Pascal Muller's Research