The Black Chamber Society

For the past few years I have been running a Codes and Code Breaking course at Ohio State. We got the idea from Chris Kennedy who taught a similar course at Northwestern, and before that at UCSD, where the course has now been pushed in a slightly different direction by Andy Kehler. It's a very enjoyable course to teach.The way we do it there is a lot of problem solving and group work, and just a smidge of modern stuff like public key cryptography. We feed the people who become passionate about the math and CS aspects into courses like Steve Lai's graduate CSE class. I am grateful to the National Science Foundation for helping support the course in its early years. Most of the students come from Linguistics or from the Security and Intelligence major. They're delightful (usually) and smart (on average). Both Linguistics and International Studies seem to get many students who enjoy learning and are good at it.

We are beginning to open this course up to be taught by graduate students as well as faculty. The first graduate instructor is the wonderful DJ Hovermale who uses new-fangled avian social media to give clues to the assignments. He also sometimes uses disguise, confederates and a thick Russian accent to make the first lecture into a coup de theatre. When I teach the class again, I'll need to up my game.

The students last quarter were so fired up by all this that they decided to keep the thing going as a student society. So we have (drumroll) The Black Chamber Society. This is immensely gratifying.  Thanks to all concerned.

DSM V

DSM V is the new version of the Diagnostic and Statistical Manual of Mental Disorders. There are some major changes that are very well covered in press stories (bipolar? children, Asperger's). It's obvious that a label such as Asperger's can affect the expectations and actions of doctors, patients, family members and anyone else who gets involved. The new DSM gives us a chance to think about some of the following:

• What happens when a doctor applies a label to a patient. Do they get access to medication, support services, behavioural treatments? Are any of these forced on the patient?


• From the reports it seems to an open-and-shut case that some children diagnosed with bipolar disorder have been receiving heavy psychotropic drugs when they shouldn't have been. This is a case where the changes in the DSM are clearly motivated by a desire to reduce the amount of inappropriate treatment.


• What about the people who currently identify as Asperger's but do not necessarily fit the diagnostic criteria for the sub-class of Autism Spectrum Disorder that replaces it?  Such people might easily have Aspergian traits, but have learned, for example, to handle social interaction with a degree of smoothness and grace. Does their identity depend on fitting into a particular medical category, and if so, is that OK?


• It's a commonplace among social scientists that labeling a person has consequences. Many mental illnesses have an associated stigma. To a degree, Asperger's has an anti-stigma, because of the wonderful things that Aspergians have often done. So it turns out that removing the label is an act that has consequences.


• In the New York Times, Dr William Carpenter is quoted as saying "“Concerns about stigma and excessive treatment must be there. But keep in mind that these are individuals seeking help, who have distress, and the question is, What’s wrong with them?”. This is interesting, but actually the DSM is not exclusively for doctors and other health professionals, it is also a resource for people who don't necessarily think of themselves as having anything wrong with them. I was queasy about using the term patient earlier on. This is why.

If you care to comment, take a look at DSM5.org The process looks as if it is very top down. Here's a representative piece of verbiage.

Anyone can submit their suggestions and ideas to the members of the work groups through the DSM-5 Web site, by clicking the "Participate" button on the upper right hand side of this screen and registering. The proposed draft revisions to DSM-5 are posted on the Web site, and anyone can provide feedback to the work groups on these during periods of public comment.

Finally, members of the DSM-5 Task Force have given numerous interviews to members of the trade and consumer media to help explain the process of development to mental health professionals, consumers and family members, and members of the public, and will continue to do so through the development process.

[my emphasis]. Notwithstanding the public comment process, I don't see a strong intent to involve. anyone who is not a psychiatric professional. I think this needs to change, or be changed.

No problems with Vampyres here, move along...

Concerned spouse: Oh, OK. It's blood, I thought it might be jam.

Spousal concern was about potential breakfast-related grooming error. Loss of blood due to shaving injury not a concern. Could she be a Jampyre?

XML and Corpora

I just got asked to advise on tutorial materials about XML for a computer scientist starting a corpus encoding project. To get started with the very basics I like Greg Wilson's Software Carpentry lecture. For more advice on how to go about building up a corpus, see Developing Linguistic Corpora: a Guide to Good Practice, by Martin Wynne and a bunch of Humanities Computing luminaries.

The project is going to work with childrens' books. That led me to find the Comic Book Markup Language provides a tool for adding analytical markup to (wait for it) Comic Books. It uses TEI, which is great, but heavy-duty.

I love it when people try to work out what I want

GigaOM (which I saw in a tweet from Matt) claims that Google is willing to do our searches without directly taking our money because it "takes its payments in a currency just as precious — user data — that helps advertisers target ads at readers". This is obviously true, but misses some of the point. The user data can certainly be used to target ads, by working out which advertisers provide the service that the reader is looking for, but that is very far from the only use. It can also be used to (sometimes) work out what people really want. There may not yet be an advertiser willing or able to offer that thing, but maybe Google (or Bing, or someone) can invent a way to provide it. This gives you data-driven innovation before you've had the innovative idea. Would it be too much of a stretch to say that Google and the others are picking up on our inventive ideas and commercializing them before we even know that we have had them? It might be fun to get outraged about this, but I'm OK with people trying to work out what I really want. Just let me know, OK?

Safety precautions for data mining expeditions

Safety precautions for data mining expeditions

Prof. S.N.L. Ingqvist, Independent Data Prevention Consultant

The following text was found on the still twitching body of an explorer found near the opening to the lair of the Lithuanian gerund. It was a clear case of data poisoning. For your own protection, you may wish to consider the advice that he evidently ignored. If he’d taken it, he might still be alive today.

Data mining is an inherently risky activity, and should not be undertaken unless you are aware of the risks. But with care it can be safe and enjoyable, even for the novice. If this is your first time, you may wish to engage the services of an experienced guide; but before taking charge of an expedition, or going it alone, there are a few simple safety precautions of which you should be aware:

• Make sure you are properly equipped. You need the stout boots of self-assured theoretical insight and the thick jacket of intellectual pride. Remember that equipment that would be able to resist data for an afternoon may not hold up so well on an overnight trip. Lighter materials may quickly become saturated with data and afford little protection.


• Make sure that someone knows where you are going and when you plan to return. Darkness can descend fast in the data mine, and it is easy to lose track of time. If you do not return to the comforting world of established theory at the expected time, your colleagues will need to alert the Data Mining Rescue, who will send out an appropriately equipped search party. See below for a list of the intellectual resources that you can expect the rescue party to bring.


• If you are trapped in an enclosed space, no matter how desperate you feel, you should not inhale until you have taken steps to be ‘above the data’. Remember that in such an environment there will still be pockets of rarefied intellectual discourse. Stand as tall as you can, and, holding your nose in the air, proceed to your safe zone.


• You do have a safe zone, don’t you? For some of us this is a methodology. For others it is a readily available source of non-toxic pseudo-data. If you are able to reach your safe zone before you are overcome by the data, the chances are good that you will find congenial company there.


• The following section, initially unreadable, was deciphered on realizing that the cuneiform script in which it was written was nothing more than an alternative writing system for an ancient dialect of AMS-LaTeX math mode: Be well versed in communications theory. It is only by clanging on the data pipe with your bit bucket that you will be able to send messages to the surface. Furthermore, if you can learn to reduce the conditional entropy of your surroundings, less data will be stirred up and you have a higher chance of surviving data poisoning.


• Once you are in the safe zone, you should wait for the arrival of the rescue party. Do not engage in debate, controversy or heated discussion, as these will rapidly exhaust your limited supply of intellectual oxygen. If a colleague panics or becomes over-excited, the kindest thing to do is to gently force them under the data, holding them down until they cease to struggle. This will increase everyone else’s chances of intellectual survival. You may be concerned about the long term consequences of this decision for your moral compass: this is quite natural. However, brain-imaging of the hypergallic gyrus (which Descartes correctly believed to be the seat of the conscience) reveals that 37% of survivors suffer few long term effects. While you may experience some degree of low-level remorse, especially in the first few days, post-trauma counseling services are available at all defense contracting firms, many government laboratories and a small but increasing number of first-rank research universities.


• The rescue party will have access to powerful data clearing tools. These may include the use of default reasoning, two-faced logic, reductio ad libitum (with or without repeats in the development section), argument that the data is a consequence of a general principle (and therefore, having no explanatory impact, must be ignored), argument from non-existent languages, argument from notational elegance, argument for argument’s sake, and (in extreme cases) argument from personal preference (de gustibus non est disputandum, some say, but for us that is just a matter of taste). If all goes well, the rescue party should be able to lead you back out into the light before permanent damage is done.


• After your experience, you may be tempted to dive back into the data immediately. Do not do this; even the most intellectually flexible require a recovery period. The medically recommended interval is two weeks for junior staff, two months for tenured faculty and two decades for distinguished university professors with a reputation to protect.

Finally, remember that there is usually nothing to fear from data. If you encounter data unexpectedly, the best thing is to ignore it.

(previously published in SpecGram)

Finding the lost symbol

Famous schlock author Dan Brown pounded through the keyboard in search of his lost symbol. After much reflection he had realized that on 28 February 2002 his keyboard had lost one of the most important symbols that it had ever had. Those royalty statements from Lisbon were looking very different. What had happened? All his royalties were down by a ratio of  200$018 to €1. Surely this must be a conspiracy which had been established at the highest level of the Catholic Church! And this wasn't just a matter of global concern and an opportunity for bestsellers ; it had happened to Mr. Brown's very own predictive word association football keyboard. The indignity! The shame! Handsome chisel jawed Dr. Brown, 5 foot 3, not Jewish, not wearing a yarmulke,  not wearing anything painful around his left thigh, honestly, couldn't think what to do. This was bathos! This was information that the reader could ever need! This was a poorly formulated parallel structure! How could he be expected to turn out well formed paragraphs? Why was he worrying about this? when had anybody ever expected this from him? What was the lost symbol, and how much was the royalty statement?

The answer from Wikipedia is that the lost symbol is called the cifrao and that it represents a sum of currency under the old Portuguese system of escudos and thousands of the escudo. The royalty statement hadn't changed, but Dan Brown's expectations had and so had the way in which the monetary sum was represented. But at least the last symbol had been found!!