Friday, November 28, 2014

TMI Friday

(TMI Alert, as per title)

As much as I love my Swash 1000 (it's Japanese-style), I have to confess that the deeply inadequate blowdry feature means that the environmental savings are less than I'd hoped.

Monday, November 24, 2014

It's hard to be afraid of everything

She touched the foam once with the tip of her nose.  It rocked slightly, and she retreated to a safe distance, where she has been stuck since.

Thursday, November 20, 2014

"Sometimes you're the windshield", or, a rare moment when programming is not all pain

Something worked today. Hold your applause until the end.

I decided to build Whatnext (the nihilistic todo list) with something called Django, which is supposed to handle all of the common website stuff, like logins and tables and sorting.  It mostly does, but it's a very leaky abstraction.  Sometimes something that is supposed to be very simple turns out to two minutes of work, IF AND ONLY IF you already know Python data structures so well that you can instantly describe how each of the following differ:
for i in dir(context): print i
for d in context.dicts: print d.keys()
for i in context["field"].__dict__.items(): print i
If you don't know (I still don't), you'll spend a few hours on that two-minute task.

Well.  Just now I decided I wanted to bite the bullet and learn "messages", which is Django-speak for those nice little announcements that pop up after you do something on a website but don't bother you and go away in a few seconds. Slightly less than seven minutes later, I had this:


How nice is that?  It's about 8 out of 10 nice, I think; it needs to be yellow with drop shadow for 9 of 10 nice, beyond that maybe rounded corners and some spirit fingers?

I'm so relieved and startled to have something be as easy as it was supposed to be.  You're not always the bug; sometimes you're the windshield.



Tuesday, November 18, 2014

Is your task management strategy more Sisyphus or Tantalus?

The premise of WhatNext, my task management program/concept/philosophical exploration, reflects the Johnson quote:
When a man knows he is to be hanged in a fortnight, it concentrates his mind wonderfully.
Given that you have a specific, limited amount of time to do anything before death's warm embrace, which mundane, trivial, or unpleasant thing should you do in the next few minutes and hours of your flash of existence on Earth?

Okay, that's a bit morbid, but lighten up: this blog post is really about whether you approach tasks as Sisyphus, doomed to forever roll a rock up a hill only to have it roll back down, or as Tantalus:
... the gods punished him by placing him in the nether world in the midst of a lake, but rendering it impossible for him to drink when he was thirsty, the water always withdrawing when he stooped. Branches laden with fruit, moreover, hung over his head, but when he stretched out his hand to reach the fruit, the branches withdrew.
I think most people perform Tantalus-style task management.  Do you make lists of things to do?  How does that work?  You go through the list, checking things off, and if you have had a very successful and productive day, you've merely gotten yourself from "behind" back to zero.  That is, the best one can hope for is to not be at a negative.  This is a miserable way to exist.  This mentality is reified in the software development process called Scrum, with the Burndown chart:


In the burndown chart, when all of your work is done, you are at zero.  Congratulations?  You have nothing.  You think you are finished and that you have reached the fruit, but instead of eating the fruit you just start looking for more fruit; it is as if the fruit sprang away from you. In response, some Scrum people have come up with the Burnup chart, where you can see your progress accummulate, and what I use professionally now looks more like this:


On the bright side, you can see all of that lovely green of completed work accumulate.  The other colors represent various stages of work in progress, and the gray the maximum amount of work the teams thought they might achieve in one "Sprint", which is our standard work cycle of four weeks.  On the dark side, however, every 4 weeks we start over:


So while you get to go as far as 28 days before seeing all progress flushed away, eventually the chart does indeed reset to zero, and you are forced to confront the reality that you too are Sisyphus, pushing the rock up over and over again.  Even if you maintain a project burnup that shows cumulative progress over months and years, eventually your project will end, and you will have to find another rock to roll up another hill.


Addressing that is beyond the scope of today's post, although I do think there is something there that Whatnext can help with, not as software per se but as an experiment in a better conceptual framework for getting things done.  As homework for that, however, I want to read this book that Marc Maron kept pushing on his podcast, The Denial of Death.  So far I've read the introduction, and while I really liked it, I can't say that it gave me any immediate ideas for task management software.

To get back to the tangible: Now that I have a minimally working task list program in Whatnext, I want to add the first feature that starts moving it from the Tantalus or Sisyphus model of most task tracking concepts to a more positive model.  If that's not a burndown or burnup chart, what is it?  I want an immediate, positive, visible indicator of progress, so that whenever I complete a task, I can see something that shows that I have spent my time wisely.

A program called RescueTime offers one idea: a "Dashboard" that shows a quantitative moral verdict on your minutes:


In this chart, red is time wasted, and blue is time well spent.  More specifically, blue is "productive" time, and you can numerically judge the quality of my time today by the ratio between the two:



This thread of thought—I beg your pardon, as I started this sentence, RescueTime, as if it was reading what I type (and, given the state of privacy and how little companies respect it in 2014, it may well have been), flashed up this quote:
“The time you enjoy wasting is not wasted time.” — Bertrand Russell
Which is exactly where I was going with this idea.  Before you can judge productivity, you have to categorize all of the ways you can spend time, but even before that, you have to understand what you mean by productivity.  My numbers for today looked much worse just now until I told it that blogging counts "Writing", which is categorized as "Design & Composition", not News & Opinion; that makes it "Very Productive", not "Distracting". What is your product?  Actionable deliverables?  Pages of product?  Clicks?  Revenue?  Acts of kindness?  A sense of satisfaction?  The ability to die in peace and contentment?  What are you being distracted from?  Being productive, or acknowledging the Grim Reaper?  Why does she always have a clock?  I'm very exciting to get into these questions with Whatnext, but I have plenty of philosophical homework to tackle first.

In the meantime, reeling us back from the abyss of nothingness to the simpler hell of management-speak, let's think about Milestones.  Or, let's call them Thresholds.  At the moment, other than blogging, I have a dozen tasks I'm thinking about for Whatnext.  If I arrange them around what they enable me to accomplish, I have a list of new capabilities and achievements instead of a list of tasks to burn through and hide.
  1. "testing".  Instead of a big handful of testing tasks that I've been neglecting, I have the opportunity to save myself from regression bugs in the future.
  2. "Toggl integration".  Instead of a messy API integration that will require me to climb up several brand new learning curves while experiencing the frustrations unique to cross-product integration, I have the opportunity  stop copying and pasting time data between websites.
  3. "Integrating task tracking lists".  Instead of the headache of merging my task lists from four different tools (Redmine, Checkvist, draft emails, Google Spreadsheets), each of which does only a fraction of what I need it to do (which is why I'm working on Whatnext at all), I have the opportunity to .... enjoy freedom from worrying which different list I should be looking at?
  4. "Preparing for a closed Alpha".  Instead of trying to scurry around and fix a billion bugs and catch up on the deeply unfun "DevOps" tasks that I have skipped to date but which are required for a stable, reliable, and safe website, I have the opportunity to share my ideas with some friends.
  5. "implementing thresholds".  Instead of wrestling to articulate a fuzzy concept of a data model and then stab in the dark at how to implement it, all which a severely limited set of tools due to my inexperience at Django, I have the opportunity to simplify how I decide what to do next.
At its worst this is just putting a falsely happy spin on unpleasent or difficult things, but I do think there's a nugget in the concept of Thresholds that is useful and positive.  My idea is that each tag could have some specific milestones (which I'll call thresholds until I get an idea which is both better and actually good), and then all tasks with that tag can be sorted as either needed or not needed for a threshold.  What could make this different from less-that-helpful paper exercises like Gantt charts and rollups and the like is if I can design an interface that makes it simple, fast, and context-specific.  In particular, I think this has to be tied into the idea that there is no two-dimensional wall chart of activities and durations; there is only a one-dimensional list and everything is sequence.  There's only one of you at a time so you can only do one task at a time so all tasks must be sequential.

It might end up something like this:

0) You jot down a bunch of things you think you need to do, and they all jumble in with the dozens of other things you previously jotted down.
1) You pick a tag, i.e., a context, to focus on.  Just stuff for your job, or just household tasks.
2) Within that context, you define one or two outcomes, triumphs, accomplishments.  Things that probably have a lot of separate sub-tasks.  Not tasks, but conditions of success.  "I am ready for my trip next week."  "My home is clear of things that I don't need which can be recycled or donated."  "All of my homework for the rest of the year is completed by captive demon."  Probably you rank or date these somehow to indicate priority.
3) Based on the tags and maybe dates and such, you are able to look at a limited number of tasks, maybe one screen full, maybe that's about 20 or 30, and you move the tasks above and below the thresholds you just defined.  Based on this, the system helps stack-rank all of the tasks.  E.g., "gather all old newspapers into paper bags" is not necessary to get ready for that trip, so if the trip is more important than cleaning out the home, gathering the old newspapers just moved below all trip-related tasks.
4) See at all times in Whatnext some kind of indication of the most important threshold, how far away it is in time: "you have an estimated 2 hours of work to complete to reach this accomplishment and have this result", and what progress is being made: "at your current rate of progress on knitting tasks you will finish those 2 hours of work in 3 weeks".

Where this gets messy: what is the interaction of tasks across different categories and contexts?  How do you stack-rank incommensurable thresholds, such as "get promoted" versus "watch all the good movies"?  What about dependencies?  But all of those things probably belong to thresholds below the five I identified above.  And building a prototypical threshold feature in Whatnext (#5 above) will make it possible for me to figure out what to do next, so all I have to do is achieve threshold #5 in order to figure out that I should probably work on #5.

Meanwhile, she is waiting patiently for you:









Monday, November 17, 2014

Programmer problems: Is "Soft Delete" evil?

This is part of what I guess will be an ongoing series, "There's one right way to do that and it's wrong."  Today it's about deleting things: what's better, a confirmation or an undo button?

I have started using a very simple and rough version of the Whatnext (my proof-of-concept nihilistic productivity planner) as my task list; it's marginally better than the Google spreadsheet I was using, but marginal is better enough when you want to eat a little dogfood.  The very second thing I did, after dancing with glee that I got something useful onto the internet in only 53 hours of work (plus 83 for getting the website back up and learning the basics of Django programming), was screw something up.  The third thing was to get annoyed because there's a confirmation step to deletion.

Why did I make it have a confirmation step?  I didn't—I followed the example for the tool I'm using, Django class-based views (I am continuing to blur any technical terminology in these programming essays so as to avoid scaring or boring).  I don't want to scrap the confirmation because, of course, I am terrified of deleting anything by accident.  Usability guru Jakob Neilsen argues that undo is more usable than confirmation.  More generally, I suspect that begging forgiveness is more usable than asking permission as long as forgiveness is easy enough to grant.  So, let's fix this by changing it from Confirmation to Undo.

Okay, easy enough, I'm plenty familiar with the "Soft delete" pattern, which is that you don't actually delete anything in the database, you just mark it as deleted and then exclude it from normal use.  And Django has a handful of packages to make it easy.  However, googling for best practices leads to this apparent ur-objection: The trouble with soft delete.
it usually ends up causing more harm than good
Similarly:
[Soft delete is s]imple, easy to understand, quick to implement and explain.
It is also, quite often, wrong.
Even more:
I just do not like this kind of design at all. I am firm believer of the architecture where only necessary data should be in single table and the useless data should be moved to an archived table.
Ugh.  I just want delete stuff without having to click twice and without risking losing something I wanted.  I want an undo, like Gmail:


(By the way, if you ever wished you could unsend an email right after you sent it, because it's only when you click send that your brain finishes figuring out what you want to say or not say and who you want to say it to or exclude, and you use Gmail, you should totally turn on undo.  I'm sure I use it more than once a week.)

I don't have time to read all of those links and reach an informed position in this debate.  I just want an undo button.  I keep browsing as a way of stalling, and see that someone else suggests yet another pattern: "the Command Pattern."  I start reading the link:
Command is a very powerful design pattern, whose intent is to encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.
Yeah, no, I'm not going to even read enough to understand what that means.  Okay, I want to finish dealing with the delete confirmation in ten minutes or less. And we are back to the classic programmer problem: should I do it the expedient way or the best way?  I think I'm going to compromise: I'm going to switch to Soft Delete for the moment, because it's expedient and a very reversible decision.  But I'm not going to build the undo yet, because that's going to take a while and I already have a task later on to figure out how to do the yellow notifications.  If I really need to undo I can duck under the covers.  And I've bookmarked all of those articles to read through and then consider which alternative to Soft delete, if any, is best for Whatnext.

I guess that's not a compromise; that's doing a quick and dirty partial fix to the immediate problem (which, if you remember, is that the delete confirmation annoys me) and punting on everything else.  In my next essay on this subject, I should probably tell you about the term "Technical Debt," because I just incurred some.  Meanwhile, go set up undo on your Gmail.

Saturday, November 15, 2014

In a better universe ...

In a better universe, putting two fitted sheets into a dryer together would cause their tangling powers to cancel out.

Against hidden passwords and masked fields on web forms

I hate this. This is bad design. I'm typing my Social Security Number into a site and it's masked. Because of the terrible epidemic of people sneaking into other peoples' homes and watching over their shoulders as they type their SSN into websites and then stealing their identities. Ok, sure, plenty of people are doing sensitive data entry in coffee shops and libraries and whatnot, but even there, I think the hassle of not being able to see what you are doing, especially when doing complicated data entry, outweighs the notional security of forcing all users to mask.  There is probably a Firefox extension to fix that.

One thing you can do in Firefox that is nice is, on poorly designed websites that prevent you from pasting in your password, reclaim the power to paste in your password.  See here.

My reward for filling out a long form (complete with yet another classic, silently truncating your entered password) was this:

HTTP Error 500
500 Internal Server Error
This server is incapable of servicing your request. Please try your request again later.
We apologize for any inconvenience this may cause.
If the problem persists, please contact a representative of ADP to assist you.

Friday, November 14, 2014

Things that have happened recently while programming

One.  I was typing in a terminal window and viewing the results of my work in a browser window.  I couldn't see any changes from what I was doing.  A common rookie mistake is to type into one environment and look for results in a different environment, such as working on the development server and checking the production server.  I'm no rookie so I have safeguards against this, like naming things "dev" and "production".  And just to be certain, I made a huge, deliberate error that would have broken things, and sure enough, I could see in the terminal window that things broke, so I could be completely sure that I wasn't typing on one server and viewing on a different server.  Do you see what just happened in that last sentence?  It took me about an hour to catch on.

Two.  I have a function that I have strongly suspected for a very long time should be recursive.  I just allowed it to be recursive.  It works better.  I feel relieved.

Three.  That function that's now recursive?  It just broke again.

Wednesday, November 12, 2014

Programmer problems: there's one right way to do that and it's wrong

I'm still stuck on that problem I discussed last week, how to get a few numbers from one place to another place within my problem.  Or rather, I've won one little learning battle but the programming war remained frozen like Stalingrad.  What I'm trying to do, task #190, is make my program so that when a user adds a book to the list of books they want to read (for example), the new book goes to the top of their book list but not to the top of, let's say, their recipe list.

In jargon terms, I'm trying to retain and pass around context, and apply that context consistently.  That context includes tags, of course: (once again, I'm blurring out all of the jargon) if I say "Books", the system should remember "Books" for as long as appropriate.  It also includes my identity: if I add a task, it should go on my list, not yours.  And the context should stick around: after adding the task, I'd like the website to remember to show me my list of tags, not yours or everybody's.

Last week I was stuck on one portion of this problem: taking the context from one web page, like after you click "Books" and "Filter", and passing it forward to the buttons on the page so that, when you add a new task, the controls to add a new task remember that you are concerned only with "Books".  Once I found the right tool (it turned out to be something called a QueryDict), I solved that in five minutes and two lines of code.  Of course, it had taken 10 cumulative hours to figure out those two lines of code, but ... they work, and they are the Right Way for my platform of choice.

But that's only covers passing the context around; I also need the system to properly apply the context when it stores and retrieves data.  And it stores and retrieves data for many reasons—when looking at the list, or when adding a new task, or when moving a task up or down, just to name a few—so I want to write the code for that once and have it work in all places.  This is called DRY, "Don't Repeat Yourself." Being the avid learner that I am, I went to find the most correct way to consolidate all of my repeated view filtering code in Django.  The instructions say to use a custom Manager:
You can use a custom Manager in a particular model by extending the base Manager class and instantiating your custom Manager in your model. There are two reasons you might want to customize a Manager: to add extra Manager methods, and/or to modify the initial QuerySet the Manager returns.
But when I go looking for examples, I stumble across an argument:
Using raw ORM query code in views and other high-level parts of your application is (usually) a bad idea. Instead, creating custom QuerySet APIs and attaching them to your models with a PassThroughManager from django-model-utils gives you the following benefits:
Although the very first comment is pure trolling.
There is no "right" way to use an ORM. Put it down and step away.
Troll aside, more googling finds that somebody agrees with the premise:
If you haven’t read that post, head over that way to hear why it’s a fantastic idea to build high level interfaces to your models, and because of limitations in Django’s manager API, to do so in QuerySet’s rather than Managers.
but has a different solution in mind:
While this definitely has the feel of slightly noodley trickery to it,  I think it starts to get at the core of what we might want in a more robust future ORM for Django: the ability to define high-level logic to assign meaning to particular collections of fields, and to preserve that logic across relations and chains.
Somebody else also agrees with the premise, but has a yet better way to do things:
But using Python mixins I've developed what I think to be a better way
(If there's one nice thing about online arguments about the best way to do a particular kind of computer programming, and there may not be, it's that everybody who wants to be read has to provide examples; nobody can just claim there isn't enough room in the margins to back up their claim.  On the other hand, you then have to read their examples.)

So the practical question is, do I want to do things the "rightest" way, which is going to mean researching this debate and picking a side, or do things the way the official documentation suggests, or do things the quick and dirty way and pay for it later? This is not a rhetorical question; being able to accurately estimate the cost of fixing it later is a professional skill which great programmers have. In my case, I'm trying to roughly balance expediency and rigor, and that's not working very well. I probably want to lean a bit back toward expediency.

Friday, November 7, 2014

Goodnight sweet Ivar

Go to sleep you little Ivar
Go to sleep you little Ivar
Come and lay your bones on the alabaster stones
And be my ever-loving Ivar

Thursday, November 6, 2014

When simple programming tasks become epic trials simply because you don't know what you're doing

I want to get some data from one part of my web program to another.  What I thought would be a trivial little task has turned into 7 and a half hours of work over the last week (counting blogging about it), which is not much shorter than the entire Lord of the Rings trilogy on Blu-Ray.  I'm going to tell you this saga because it is a very common happening with computer programming: you think you are (metaphorically) popping out for five minutes to drop your Netflix in the mailbox and grab some Mountain Dew and realize, days later, as you dodge orcs and hide from Nazgul, that you are guiding two hobbits with some Fingerhut ring and a stalker problem across Middle Earth and into Mordor.  This phenomenon plays hell with estimation, drives non-programmers batty, and goes a long way to explaining where there can be a 10x or 100x difference in productivity between two programmers.  We're going to go through my sad little example to try and give you, a non-programmer, a helpful peek into the abyss.  Don't worry, I'll blur all of the computer code.

Let's begin with a trailer featuring a scene from the third movie, when Sam and Frodo are dodging Shelob, the giant spider.  In my version, Frodo is played by the number 28, and Sam by 29.
<QueryDict: {u'notes': [u''], u'csrfmiddlewaretoken': [u'JJUhgrMzA3gtsMPnmzUsJYn4J6doONt4'], u'name': [u''], u'tags': [u'28', u'29']}>
See them stuck in Shelob's web?  I try to get them out of there using this magic incantation:
(Pdb) print self.request.GET["tags"]
And all I get is:
29
Obviously Sam can't go on without Frodo.  Our story cannot proceed.  Disaster!  Or, to abandon the increasingly tortured and confusing metaphor, I'm trying to complete Task 154, which visually is this:


This is a task from WhatNext, the nihilistic task management website that I've been progamming as a hobby, in between the knitting.  It's a task that has completely stalled progress for, as I've said, a week.

Our story begins with this innocent-seeming web page.  This is an example of the WhatNext task list:


See the little up and down arrows?  When you click, the task should move up or down.  And that works.  But.  I also have these nice tags, so that you can look at only a little bit of your list:

And I need to keep track of those tags for almost everything, or else things go wonky.  Which means, in the grossest terms, that that bit highlighted in the red box needs to go down where that red arrow is pointing.  AND IT WON'T GO THERE.

There are two kinds of programming: programming that does something never done before, and the other kind, re-inventing the wheel.  In a sense all programming is the first kind, because if something already does what you need, why not just use that?  There should never be a need for the second kind.  But in practice there are obstacles large and small.  For example, the program you need may be very expensive and you only need a fraction of what it does, so you take a cheaper program and try to extend it a little bit.  Or you have a Mac and the program that does what you need is only for Commodore 64.  This dilemma applies when you choose a program to use, and it applies when you are programming.  All programming is done on a foundation of what has come before; you don't typically rewrite an operating system, or email, or the alphabet.  And when you are creating a data-backed website, you are writing the same kind of program that everybody's been writing for two decades, which is only a slight change to client-server programs for decades before that, all the way back to mainframes and terminals.  So most of the idioms and patterns and best practices have been codified long since, in the form of standards but particularly in the form of languages and platforms and libraries.  One need only choose and master a platform in order to start out, in theory, 90% finished, with only the particulars of your situation remaining to be programmed.  So programming is typically not inventing a new solution to a problem; it's figuring out the most efficient ways to apply existing solutions to old problems for new customers.

Which is why it's so unbelievably frustrating to get stuck trying to do something in Django so obvious and common as taking some parameters from the incoming URL and sticking them in the outgoing URLs so that the user does not lose their metaphorical place when they click on things.  There are some best practices to handling URL parameters, to prevent problems like mangling the spaces between words, or allowing a Ukranian rebel hacker to steal your credit card and denouce the West on your home page, but those just make it more likely that Django (the platform I've chosen for my program) would have this functionality built it. Which means that I've spent most of the 7+ hours googling for examples that are close to my problem, solve a piece of my problem, but don't quite solve all of it.  Django provides many, many ways to do anything.  Which means it doesn't provide a right way to do something, only ways with differing degrees of appropriateness.

Let's get specific.  First, I have to capture any data the user provides in the URL.  That is, if the incoming URL says that tags 28 and 29 are in use, I need to store "28" and "29" somewhere.  Then, I have to make sure they are really plain numbers, and not some cryptic bit of code that tricks my computer into divulging the launch codes.  Then I have to prepare them for the outgoing URLs, which in this case means turning them into the sequence of characters &tags=28&tags=29.  And then I have to stick that sequence of characters at the end of all of the URLs on the page, such as the "Move Down" button and the "Move Up" button.

I know how to do almost all of this. I've done it before in this program, everything but sanitizing the input, and I've glimpsed bits of that so I'm confident I'll find tools for it in the platform.  Nothing here should take more than five minutes, so I'll round up and say the whole task ought to be done in an hour.

First, I have to catch the data. The standard place to capture incoming variables in Django is urls.py.  However, this makes it clear that I can't use it because the technique is not appropriate for parameters that "chang[e] the way the resource is displayed".  Another programmer might decide to go ahead and use this technique anyway, and that might work for them, or they might hit a different snag later.  Or it might work now but then cause a problem for the next programmer in a few years, when something else changes.  I'm going to play it straight and try to find the preferred way to do this.

So I look for another way to catch my 28 and 29, and find it, but when I put that code into my program, nothing happens.  Nothing happens is probably the second-most common outcome of any change to a computer program, beaten only by error messages.  Why doesn't anything happen?  I Google, and find another question along the same topic, and the answer should clarify some things for you:
... they are at the class level and are class variables. As for the NameError, where are you trying to do year = self.kwargs['year']? You should be doing it in a method, you can't do it at the class level. 

I think that that means that you can't use these particular commands in the main body of a "class", which is a particular type of mini-program within my program (which, let's recall, is itself inside of a another program, called Django, which is effectively inside of another program, called Python, which ... reminds me of a movie called eXistenZ.

Do you remember that movie? SPOILERS! It came out the same year as the Matrix and The Thirteenth Floor. In the Matrix, it turns out that SPOILERS our reality is just a computer simulation. In The Thirteenth Floor, researchers get stuck in their computer simulation, but when they get back to reality it turns out that they in turn are just part of SPOILERS a computer simulation. So that's three layers to the Matrix's two. Add a layer of complexity, shrink the box office by an order of magnitude. In eXistenZ, you simply lose count of the layers of reality, so it didn't make much money at all. In Python, each bit of program tends to run inside of another bit of program, and you can interrupt the program and run a command to show you all of the layers of program that are in effect, so that you can track down problems. Here's what you get with my program at the point of error:
(Pdb) where
  /usr/lib/python2.7/threading.py(783)__bootstrap()
-> self.__bootstrap_inner()
  /usr/lib/python2.7/threading.py(810)__bootstrap_inner()
-> self.run()e
  /usr/lib/python2.7/threading.py(763)run()
-> self.__target(*self.__args, **self.__kwargs)e
  /usr/lib/python2.7/SocketServer.py(593)process_request_thread()
-> self.finish_request(request, client_address)e
  /usr/lib/python2.7/SocketServer.py(334)finish_request()
-> self.RequestHandlerClass(request, client_address, self)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/core/servers/basehttp.py(126)__init__()
-> super(WSGIRequestHandler, self).__init__(*args, **kwargs)e
  /usr/lib/python2.7/SocketServer.py(649)__init__()
-> self.handle()e
  /usr/lib/python2.7/wsgiref/simple_server.py(124)handle()
-> handler.run(self.server.get_app())e
  /usr/lib/python2.7/wsgiref/handlers.py(85)run()
-> self.result = application(self.environ, self.start_response)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py(67)__call__()
-> return self.application(environ, start_response)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py(206)__call__()
-> response = self.get_response(request)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/core/handlers/base.py(112)get_response()
-> response = wrapped_callback(request, *callback_args, **callback_kwargs)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/views/generic/base.py(69)view()
-> return self.dispatch(request, *args, **kwargs)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/braces/views/_access.py(64)dispatch()
-> request, *args, **kwargs)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/views/generic/base.py(87)dispatch()
-> return handler(request, *args, **kwargs)e
  /home/ao3/ao3/local/lib/python2.7/site-packages/django/views/generic/list.py(152)get()
-> context = self.get_context_data()
-> /home/ao3/ao3/whatnext/views.py(96)get_context_data()
return context

Each -> is another layer of context, a bit of previously existing program that is part of the platform of Django and does some essential bit of the process of serving up a database-backed web page. So part of my time is spent checking the most superficial layer, and maybe one or two layers below that, to see what's going on and look around for where 28 and 29 might be. But I also spent a few hours (not included in the 7.5) getting the development site working on my laptop, so I could work on this in coffee shops and airports, and then on my laptop this particular debugging tool wasn't working right, so that was another thirty minutes fussing. And as part of getting the site running on my laptop, I went to use a backup copy of the data to put on my laptop and realized that I didn't have automatic backups running, so that was another thirty minutes to get that going, and if I'm conscientious, another 30 minutes in a week or so to make sure backups are actually happening.)

So, three hours of parathentical work later, I figured out from reading the internet that the command that retrieves 28 and 29 has to be used inside of a method, which is a mini-program inside of the mini-program. So, that's a hurdle well cleared. Now, I have to figure out which mini-program in my mini-program is the right kind.  That is to say, of the many mini-programs that Django developers have added and made idiomatic, I have to figure out which one they intended to be used in this circumstance.  In this case, I *think* it's get_context_data, because of yet another helpful question.


Also, at this point one is generally tense, bored, and frustrated, and a little stupid, so something like "Don't mix *args and **kwargs in call to reverse()!" becomes very funny: I guess although *args and **kwargs get along fine in normal conditions, in reverse() they fight and, since the **kwargs are bigger and have more asterisks, they tend to eat all of the *args.  (That's a kwarg to the right, although it may actually be Kwarg itself.)

So I think I know which mini-program I should put my commands in to get 28 and 29, but then I read that Django’s generic class based views now automatically include a view variable in the context, so maybe I don't need that first step at all, that's already done automatically and I can skip to the second step, which if you'll recall was to make sure they are safe, innocent numbers? Except that, before I can move on to that, I get stuck because when I ask for my data in exactly the right place, using what I think is the right magic command, I just get one number, not both. You may remember this scene from the trailer.

(And, to be completely clear, 28 and 29 are just stand-ins for many possible numbers, depending on what a user enters.  I'm just assuming that, if I can get 28 and 29 through to where they need to be, it will work for other numbers.  But I'm lucky that I picked two numbers, not one, to be my sample data, because if I'd picked just 28, for example, then I never would have noticed that a hypothetical second, third, or later number would just disappear.  That is, I wouldn't notice until weeks later, when I had already been using my program and any task that had one tag would move just fine, but any task with two or more tags would, when moved, go to the wrong place, and how many times would that happen before I noticed, much less diagnosed the problem?  What other potential problems am I not uncovering now because of my choice of 28 and 29 as my working test numbers?)

The reality check that keeps creeping into my brain is, surely many people have solved this exact problem before me, using this exact program?  I'm not doing anything special.  Why can't I find any giant's shoulder to stand on?  So this becomes not a programming problem, but a problem of finding the right query to lead me to a previous example.  Finally, I try "django get list from querydict", which leads to a clue (emphasis added):
QueryDict.iteritems() uses the same last-value logic as QueryDict.__getitem__(), which means that if the key has more than one value, __getitem__()returns the last value.

Which is helpful, because it's talking about exactly the place where I'm stuck. And, more importantly, it's clued me in to the fact that the tool I'm looking for, the thing built in to Django to make this problem easy, has been right under my nose the whole time. It's QueryDict. I haven't solved my problem, but I've found the right tool to use, so I guess I'll just go and read up on that now. And this, amigas and amigos, is what may be happening when your programmer goes off to do something that even they think is simple but, days later, when you ask how it's going, they just snarl and throw cheetos at you. 

This story also illuminates the discrepency in performance between different programmers.  I know in fairly specific terms what I want to happen, but have no idea how to make it happen in this particular platform, so I'm spending all of my time learning the platform, painfully.  A "1-hour" task is shaping up to be 10 hours or more.  Another programmer might not know this platform but might have developed much stronger instincts for data-backed web idioms, and would find the tools they needed much more quickly.  When I complete Task 154 and commit the new code changes, I expect there to be maybe 10 or 20 new lines of code.  Someone completely familiar with Django might have solved this task in literally the time it takes to type out those lines, no extra thought or research required.  Someone fairly familiar with Django but sloppy might have solved the task in 5 minutes but also have added in the bug we spotted, where moving a task with two or more tags puts in (silently) in the wrong place, because 28 and 29 turns into just 29 but they don't notice.  All I know at this point of Task 154 is that I'm too ignorant even to estimate how much more time it's going to take me.

Good rappin' at you, amigos and amigas.  If you take nothing else from this blog post, I hope you take with you a wonder at why you read this far.

Monday, November 3, 2014

San Franciscans - go see the Touré-Raichel Collective this Saturday night

I saw them last time they were in San Francisco and it was the most pleasing live music I've experienced in a very long time.  Buy tickets right now and then go listen to a little bit to whet your ears.  My favorite part from the live show was when Raichel (piano) and Touré (guitar; he's the son of Ali Farka Touré) had a musical conversation: you can sort of imagine Dueling Banjos but, instead of challenging and repeating, it sounded to my barely musically educated ears like they were having a conversation, or even telling jokes, and it sounded lovely even when I didn't get the jokes, but a few of the jokes I understood, or imagined I understood, for example an arpeggio to be a punchline, and I laughed a little bit too.  And then the calabash player and the bass player did the same thing and were just as good and funny.

(Also: Seattle on Nov 15 and Toronto on Nov 21)

Sunday, November 2, 2014

Hat 38 is done

Thanks to the bizarrely warm summer here in San Francisco, not too many heavy wools hats have been warranted.  But this one is done and today was a bit chilly.



The Ravelry project.