Friday, May 23, 2014

Web programming is mostly server administration

I sort of lost track of what to do next with my web site project, so that means time for a blog post.

I have two big goals: restore's old content into a modern website, and build a new task tracking system to experiment with various ideas.  I'm sort of stuck, and my pretty list of working tasks has turned into a tangle:

The hierarchy broke down within Task 16 (Set up production Django) but I'm pretty sure that if I mapped it all out I'd have some loops.  (Setting aside the biggest loop, that I need to build a production-calibre, Enterprise-grade task management system to keep track of the tasks necessary to learn how to build a production-calibre, Enterprise-grade web site.)

I got part way into recovering my old web writing, enough to reassure myself that it was totally practical and also that most of my old writing follows Sturgeon's Law ("ninety percent of everything is crap").  Then I got hung up on the detail that, while I could recover the original publication date, Django-CMS was not eager to help me do anything with it.  I could see the data in the database, and I could see how to put a variable into the html template that gets displayed, but I was blanking on what lay between these two end points.

So I decided to take a break on the coding and go shore up some foundations.  I was running the quick and dirty Django development site, on SQLite.  I'm much more comfortable with PostGreSQL, and working on code without version control is like going commando in a baseball game: you'll get away with it for a while but when a ball finally bounces into your crotch, it will turn out not to have been worth it at all.  And maybe it's more agile to get a production-ready system launched and then roll out improvements than to build the whole thing on a temporary dev server before launching anything.  So it seemed like a good time to work toward four intertwined goals:
  1. production system that runs on a sturdy database, starts automatically on reboot,
  2. full version of the system running in a local VM
  3. everything into source control
  4. automated deployment of the site across different hosts
Going local is easy enough with VirtualBox, although I don't care to see the word Oracle appear on my screen.  Everything into source control is kind of stalled because I decided that I made a mistake going with git over Mercurial, with the tipping point being 10 things I hate about git, which did not have Julia Stiles or Heath Ledger but did have diagrams of the kind of version control I'm comfortable with (Julia):

And what git offers (Larry Miller?):

Git is basically aimed at one user, and so it's pretty quirky.
The most spectacular example of this is the command “git am”, which as far as I can tell, is something Linus hacked up and forced into the main codebase to solve a problem he was having one night. It combines email reading with patch applying, and thus uses a different patch syntax ...

Since I'm not going to integrate thousands of patches from hundreds of contributors over an enormous code base, or learn Finnish, this learning curve is a lot of cost for no benefit.  I do still want to master distributed version control, so time to get going with Mercurial.  However, Github is really useful both for the central and stable code hosting and for the sharing.  However however, Github appears to have some toxic culture issues.  Still not sure what to use instead.

Back to goal 1, production hosting, I've been reading various installation instructions for Django-cms, such as How To Install and Configure Django with Postgres, Nginx, and Gunicorn. But after four or five rounds of this, getting various errors and tracking them down, many of them going back to minor discrepancies in the instructions, or conflicts from trying to apply two different, overlapping but individually incomplete (for my purposes) instructions, I'm getting fed up. This kind of mediocre documentation is what led me to rewrite the OpenACS installation instructions and get involved in that community. And my goals here are not weird; I want a production environment for my website, and a development environment, and a straightforward way to move code and data between the two, and an easy way to spin up more environments as needed. This should be utterly standard and common; there is no reason to have a billion different ways to do this and all of them obscure. OpenACS was providing mostly adequate solutions to these problems (how do I promote code from dev to test to production? How do I move code around when it includes database changes?) in 2001. Why is there nothing better yet? And can I please get a damned diagram of Postgres, Nginx, and Gunicorn, and an explanation not only of why I end up with this in my directory structure:
but of why anybody thinks that's even okay?

And then I found this:

which comes right after this:
But struggling to configure nginx is just a waste of time. I’ve found many partial guides to Django deployment but haven’t found any single, recently updated resource that lays out the simple, Pythonic way of deploying a Django site in production. This post will walk you through creating such a set up.
Okay, I am renewed.  New task list:

1) set up a production-ready server on my desktop VM according to these instructions
2) throw it away
3) Learn Mercurial and find some minimum offsite backup procedure
4) Set up production on a plain django-cms site
5) Manually fix the home page and put up a newer picture of Kona
6) Publish one good old article (if there is one! Nothing about that bastard Rumsfeld, no matter how damning it is)
7) Figure out all of the CMS stuff (menus, graphics, minimalizing the look, etc) on dev and push it out
8) Go back to recovering blog content and the Songs not in 4/4
9) Build the ultimate task tracking system

The only problem is that I now need to get this list of nine things into my task tracking system, which is annoying, and makes me want to build the ultimate task tracking system.

Monday, May 19, 2014

Progress, and meta-progress, on my blogging/coding/project management project

I got (some) of my old blog entries to (partially) load into a django_cms development server:

which mostly served to show that probably nobody will ever want to read most or all of these posts ever again.  But I won't let that stop me!

I got passable results using Redmine to break out the work and then to track it:

This is quite workable, although it doesn't meet a lot of the requirements I would have to use it seriously.  For example, notice that the green progress bars only show progress for tasks with subtasks.  Finishing a task without children does not produce a nice green bar.  Nor it is scaled by estimated effort.

I had planned to preserve the old URLs, hacking something into django-cms to catch and map all old addresses to the newly imported content, but after realizing that Google has wiped all traces of my site since it's been down for years, that became a much lower priority.  I guess I should figure out how to search Google for links to my site, to see if any are left, but that would be pretty sad.  I will, however, have to think about preserving the links since those are recent and getting some traffic.

One thing that worked, somewhat, was the integration with Toggl via the Chrome plugin, although I'm still not sure if that's elegance or pure kludge or both.

I guess kludge, because none of that data sent back to Redmine in any way.  But anyway, every step brings me closer to unleashing poorly written political tantrums from 2003 back onto the internet. 

Guess what? Another hat.

Hat #36.  For some reason this pattern never totally stuck in my head, so I was always going back to my little slip of paper, and when I did think I mastered it, I screwed up one of the cabling rows (don't look too closely at the top third).  Also, in person this yarn looks a bit like dryer lint, and once you see the connection, you can never unsee it, so I can't give this hat to anyone who reads my blog.

The pattern is in a lovely little book which doesn't name any of the patterns.  For a continuous pattern, in repeats of 12 stitches, 8 rows:

Row 1: p2, k8, p2
Rows 2, 4, 6, 8: knit
Row 3: C4F (cable four front, meaning, slip two stitches to the cabling needle, keep it in front, knit the next two stitches, then knit the two stitches off the cabling needs), k4, C4B (as C4F but keep it in back, not front)
Row 5: k4, p4, k4
Row 7: k2, C4B, C4F, k2

With the back-to-back cabling, I ended up with some little holes, somewhere on the spectrum between lace and mistake.  Definitely one pattern that I am uncomfortable learning from a book without someone to ensure I'm doing it right.
The Ravelry project.

Sunday, May 18, 2014

Lovable losers?

Normally I wear the hat of whichever of my two teams won yesterday.  And the Tigers won and the Dodgers lost.  But ... the random number generator of fate put up a hell of a game yesterday:

The Dodgers fell behind 7-0 early, then 9-2, got as close as 9-7, and then gave up another 9 runs.  Second backup catcher Drew Butera appeared in relief for the second time in four days.  Yesterday morning the Dodgers were 8 and 1 vs Arizona, beating them on two different contintents and with a 53/28 run differential.  Today it's 8-2 and 60/46.

So, out of solidarity and because my individual actions as a fan surely make as much of a different as voting, this morning is Blue.

Friday, May 16, 2014

Nick Tortelli's adventures in American Health Insurance

Sometimes I get a sore neck.  Then I walk around like Nick Tortelli for a few days or a week.

It happens maybe three or four times a year, for maybe the last decade. When it happened last winter, the pain actually put me in bed for a day or two until I saw a doctor and got a prescription for a muscle relaxer. Don't get old.

There's a physical therapy place two blocks from my apartment where I've been a few times and it seems to help. After I started going there I didn't have full Nick Tortelli syndrome for months. Of course, it's usually three or four months apart, so that proves nothing. Still, it seems to help, but I haven't been in a while. This is because my insurance company only approved four visits. (Initially the UCSF doctor prescribed a UCSF physical therapist, without asking me what I wanted to do, but that's probably just because they are all trained by default to put their employer's interest above that of the patient.) In order to get more visits paid for, I have to see my doctor again. *sigh*

Meanwhile, I got a bill from UCSF on an unrelated matter that should be 100% covered by insurance (I have GEHA). I looked up the procedure, the hospital, and the name of the doctor on the bill, and it's all 100% covered by GEHA. It took 20 minutes on the phone (I guess I should be grateful that it only took this long, and that the wait was only a minute or two, despite the warning that they are experiencing "unusually high call volume"?) to establish that I was charged the out-of-network rate, because the specified doctor is not listed with GEHA under the tax ID number used to bill. The mere facts that the doctor is listed in GEHA's directory, the hospital is listed in GEHA's directory, and the procedure is listed in GEHA's brochure as 100% covered are phantasms. So it's my responsibility to call UCSF and tell them to (I'm not making this up; I asked what exactly to tell UCSF and wrote it down) submit a W-9 for the "provider" (no doctors or nurses in insurance-land, just providers of services) and then to re-bill GEHA.

And as someone with a full-time job and health insurance, I'm one of the lucky ones.

Come on, Loretta, we're going to go establish single-payer.

Daily affirmations for a teleworker

My passcode will be confirmed.

I will not be the only caller.

I will be joined to my conference in progress.

Thursday, May 15, 2014

Today's saga of recursive task breakdown

I went to click the timer to start working on my website and, thirty minutes later or so, I hadn't clicked the timer yet.  Here's what happened as best as I can reconstruct it from my already-decaying memory:
  1. Start to browse to whatever cloud project management tool I'm currently experimenting with, so that I can select a task and click the Toggl button.
  2. Realize I can't remember the name of whatever cloud project management tool I'm currently experimenting with
  3. Browse to my own (temporary) website to look at yesterday's blog post, which included a graphic of all of the cloud project management tools that Toggl integrates with via the Chrome button
  4. Find Trello
  5. Decide that, since Trello doesn't support recursive task trackingmore precisely, Trello doesn't support hierarchical task lists, in which each task can have another task as a parent and this tree can go indefinitely deep—I decide to go through the other cloud project management tools that Toggl supports to see how many do
  6. Look at one or two other Toggl-supported tools, which don't support hierarchical task lists.
  7. Forget that I decided to go through the list of Toggl-supported tools and google for "cloud-based project tracking with subtasks", click a top hit.
  8. Find GoPlan, read about GoPlan, sign up for a GoPlan trial, set up some hierarchical tasks, feel ennui
  9. Remember that I'm looking for a Toggl-supported tool, look at the graphic again, see Redmine
  10. Read about Redmine, including that it's vaguely inspired by Trac, which I liked, and see that it does support hierarchical tasks.
  11. Log in to the Redmine demo and create some hierarchical tasks
  12. Cancel the GoPlan trial.
  13. Discover that Redmine forked to ChiliProject over management issues
  14. Wonder if I should use ChiliProject instead, for either ethical or practical reasons
  15. Read that the driving force behind the fork left:
    "I won’t be leading ChiliProject but that doesn’t mean it’s stopping. With Holger Just stepping up as the Project Lead ..."
    which probably means that it's stopping.
  16. Read another link that suggests that Redmine shaped up after the fork, and that the general community stayed with Redmine through the whole thing
  17. Try to find a hosted instance of Redmine.  Decide I don't like HostedRedmine, for no clear reason.
  18. Wonder if, or maybe stumble across a page stating that, Digital Ocean, which I'm already happy with after a few days of using it for the development work, offers pre-built virtual hosts of Redmine
  19. See that it does 
  20. Set up a Redmine instance on Digital Ocean in about a minute
  21. Cleverly choose to use the uploaded SSH key for added security and convenience
  22. Browse to my new $5/mo Redmine site
  23. Try to log in
  24. Realize I don't have the password
  25. Check and see that I need to log in to the console to get the password
  26. Try to log in to the console and fail
  27. Realize that the SSH key it's using is from my laptop, not my desktop
  28. Get up to retrieve and turn on the laptop, and realize that I never clicked a button to start tracking my time in Toggl

Wednesday, May 14, 2014

More on recursion and work breakdown, or, how do you get to the dog store to get more poop bags if you run out of poop bags?

tl;dr:I've been experimenting with Pivotal Tracker as I work on rebuilding my personal website, but it doesn't handle recursive work breakdown well enough and I'm going to move on to the next task tracking tool. 

Here's what part of my task list for my new website looks like:

Rebuild personal website
  - Select Tools
    - Pick Language

      - Long story short, it's Python
    - Pick Platform - Django

      - Long story short, it's Django
    - Pick CMS
      - Make a list of possible CMSes
      - Look for easy ways to rank CMSes
      - Try out all of the candidate CMSes on their demo sites
        - Create a list of test tasks
        - Try all test tasks on each candidate CMS demo

  - Migrate everything from the old
    - Get the most recent backup is
    - Figure out how to preserve all of the old URLs
    - etc 

I got django-cms working on a virtual server in the cloud and decided that picking the absolutely best Django CMS was not worth the time, so that simplified the "Pick CMS" task.  Now I need to do some things like get the dev server working on Postgres instead of sqlite, set it up to run automatically on reboot, set up users and permissions, get all of my customizations into source control and rebuild dev from source control.  I went to put that into Pivotal and couldn't decide if each of those was a story, or a task within a story, or what.

I want to experiment with deep tree tasks because that's how I tend to think about tasks.  Before I can do this, I need to do that.  Before I change 200 documents, I need to ...
  • back them up
  • See if there's an easy way to do search and replace in whatever tool I am using
    • If not, see if there is a better tool I can get
  • Get Microsoft Office to stop crashing every three minutes
    • Get auto-save working every 2 minutes
  • Eat something
    • Cook something
      • Clean the kitchen
        • Empty the sink
          • Empty the dishwasher
            • Run the dishwasher
              • Buy more dishwashing soap
                • Go to the store
                  • Take the dog on a walk to the store
                    • Refill the poop bag holder
                      • Go to the pet store to buy more poop bags
                        • Take the dog on a walk to the pet food store
                          • Refill the poop bwait a minute...
So obviously one risk of this approach is that you can get trapped in a loop and never get up.  Or you can wander off and never come close to dealing with the original task.

To deal with this, you can figure out how long each branch might take and do some cost-benefit thinking.  Of course, that takes time, so you have to think about how much time to spend thinking about how much time to spend.  And that goes off into its own infinite recursion, so some amount of guessing and satisficing is essential.

Assuming all that's sorted out, I'm interested in tracking and reporting on this recursive breakdown.  If a task is estimated at "big" or "8 points" or "2 hrs", and then gets broken into three other tasks, some of which are broken out further, can you learn anything by comparing the original estimated time for the parent task with the total actual time of the child tasks?  Should the sub-tasks be estimated independently?

So I guess my requirements for my task tracking tool now include:
  • It stores an ordered list of my tasks
  • I can break a task down into sub-tasks and retain their relation to the parent task.
  • The tool integrates with Toggl, so that I can pick a task from the tool and then track my time with Toggl.
I have a lot of options.

Trello says that it's "infinitely flexible" so I guess I'll try that next.

The lure

My image of myself includes strong resistance to marketing and advertising.  I like to think I'm pretty skeptical.  But this got right past my defenses.

Thursday, May 8, 2014

The Sit/Stand/Walk/Flowerpot Desk Index

Here are all of the posts about the Sit/Stand/Walk/Flowerpot Desk:

Prerequisite for a dream

The Basics of the Sit/Stand/Walk/Flowerpot desk

Something else that isn't a hat: a Counterweight Cozy

How to build the Sit/Stand/Walk/Flowerpot Desk

The design challenges and failures of the Sit/Stand/Walk/Flowerpot Desk

An addition to the Sit/Stand/Walk/Flowerpot desk

Another upgrade to the SSWF desk: Third Monitor


Another upgrade to the SSWF desk: Third Monitor

Despite have two large monitors on the desk, I often want even more display. So, I added an old TV as the third monitor. Mounting was very straightforward with the Vitek VT-LCD/PMT pole mount, $55 with shipping. It bolted onto one of the front vertical pipes of the desk, and attached to the VESA screw holes on the back of the monitor. It's rated for a 15kg monitor, which is more than the main 30" monitors on the desk, but I wouldn't necessarily trust the tension of the adjusting bolts to hold anything much heavier than the 15" TV it's got now.

So now I have a third monitor which is useful for baseball or for showing my current task list and Toggl time tracking. I'm still tweaking the details of all the peripherals, which are mostly fine but very frustrating in the tiny details. Currently there are two computers, one Kubuntu and one Windows (after a few years of VMWare, I'm much happier without it). The keyboard is on a KVM switch between the two; the switch is pretty reliable but Windows takes about five seconds to accept the keyboard each time, compared with one or two for linux. Each computer has a dedicated mouse. They both share the same speakers (since I only have one set of ears, and having the sound switched meant I couldn't type on computer A while listening to computer B). The big monitor on the right is always Linux; the big monitor on the left switches between Linux and Windows, and the new third monitor is always Windows. The two big monitors are on their own power bar so I can turn them completely off; the TV has regular power since, unlike the big monitors, it doesn't turn itself back on when power is restored. This also lets me turn off the big monitors while still watching the little one. However, I couldn't get sound to work on the little one because of its buggy HDMI implementation, in which, as far as I can tell, it won't take sound over the HDMI cable but also won't play sound from other sources while the HDMI cable is plugged in.

And the mouselet me tell you about mouse problems. I had a Logitech and a Microsoft, but the Microsoft mouse kept freezing and glitching. I wasn't sure if that was mouse interference or just Windows, but I switched to a different model of Logitech mouse for Windows and now they both work fine. Also, the old Microsoft mouse didn't have much range when the receiver was plugged into the back of the Windows computer, so I plugged it into one of the monitors and extended that to the computer (which means remember which monitor is plugged into which computer), but it still didn't reach back to the coach. And when I turned off the monitors to watch the little monitor, that killed the mouse. Which actually isn't a real problem since, if I'm going to step away from the computer but leave baseball playing, I will put the baseball on the big monitor so I can see it from across the room. But still.

These are the things I have been fussing with rather than world peace, but on the other hand many other people have tried to cause world peace and we don't have world peace but I have three monitors and the Tigers have the best record in baseball.

(All posts about the Sit/Stand/Walk/Flowerpot Desk Index