Saturday, June 28, 2014

Looking for names

Not-yet-Milo and I skim the better parts of the Tanakh for possible names.


And Saul's sons were Jonathan and Ishvi and Malkishua ... And the name of his commander was Abiner son of Ner, Saul's uncle
... And Kish, Saul's father ...

And David was the son of this Ephrathite man from Bethlehem in Judah named Jesse, and he had eight sons ... And the names of his three sons who went to the war were Eliab the firstborn and the secondborn Abinadab and the third Shammah.
Obviously, we didn't find any he liked.  And why are half of these names perennially popular several millenia later, and the other have gone?  Was Abinadab the Aiden of 320-310 BCE?

Wednesday, June 25, 2014

To: Milo

To: Milo
From: Your parents, family, and dear friends
Date: 23 June 2014 (25 Sivan 5774)
Re: Becoming part of community

Milo, we were overjoyed to introduce you to the world on your eighth day of life. Below are some thoughts from David (Cantor Frommer), your parents, and others whom we hope will be important people in your life.

Becoming Part of the Covenant/Receiving Your Name

Ruach avoteinu v’imoteinu, kayeim et hayeled hazeh l’aviv ul’imo. V’yikarei sh’mo b’yisrael Yosef Ben Basha Leah v’Yoel Shmuel. Yismach ha’av b’yotzei chalatzav v’tageil imo bifri bitna. Zeh hakaton gadol yih’ye. K’sheim sh’nichnas labrit, kein yikaneis latorah ul’ahava ul’ma’asim tovim.
May the spirit of our ancestors sustain this child through his parents’ loving care. Let him be known among our people Israel as Yosef, son of Basha Leah and Yoel Shmuel. May his mother and father rejoice in his creation and in the growth of his body and soul. May his name be a source of joy to him and may it inspire him to a life of Judaism, of learning, of love and of kindness. And let us say, Amen.
Milo Cousens Aufrecht – Yosef ben Basha Leah v'Yoel Shmuel – your name deliberately includes your connection to the Cousens and Aufrecht clans, and each of those families offers a rich heritage that was marked by the events of 20th century Jewish life, by community, and by a commitment to intellectual curiosity and social change. We are excited (and sad) to share with you the Aufrecht German heritage, history, and tremendous tenacity, and the Cousens' role and sense of responsibility to Detroit labor life and Jewish life.

Your first names are for your great-grandparents, your Grandma Bonnie's parents, Miriam and Joseph Cohen. Your ema had a very special relationship with them. They prioritized family and community; their home was the center of their large families' gatherings in the 1950s and 60s, and they each served in Temple and home association leadership roles. They were athletic; they bowled and swam. They traveled and loved adventure. Joseph particularly was intellectually curious, trying to learn about all kinds of topics and keeping up with technology as it changed. Their house was filled with books, with non-fiction and classic novels. Joseph read the paper every day, complaining and commenting (like your father). When your ema was sixteen, Miriam had a stroke that incapacitated her. For almost thirty years, Joseph took complete care of Miriam without resentment. He took over the cooking, he managed their social schedule, he lifted her and put her into bed. His patience for her care and love for her were equaled by the care with which she had built a home for him earlier in their lives.

Miriam and Joseph were the people who made your ema feel safe, who supported her growth in her early years, who took her swimming and told her stories. Some of her earliest memories are of being with them or of being in their home. Joseph taught your ema and Uncle Jeffrey to play checkers and cards and told them bedtime stories of Brer Rabbit. Miriam brought your ema to art classes and made special birthday celebrations. They lived through her mid-thirties, and she spent much of her twenties and thirties with them in their house in Florida, helping her Grandpa take care of Grandma and visiting with Grandma, or running errands, cleaning, managing finances, just trying to support them. They were an anchor in her life. May you have their love of community and family, Joseph's generosity, patience, and love for his children and grandchildren, Miriam's sense of community leadership and responsibility, their desire to explore and to support change in the world.

Blessings

from your parents

Milo, may your heart be open to wisdom from all sources, from your community's tradition and from traditions of the world. May you use this wisdom to learn, to teach, to protect and to take action. May you be given long days and years, a life of abundance and honor, a life in which the good wishes of your heart, will always be fulfilled. And let us say, Amen.

from Rabbi Carla

Welcome to a family that values really yummy vegetarian food; thoughtful and well-researched debates about social justice issues from labor to privacy; dedication to ambitious projects like a do it yourself treadmill desk and climbing to the top of every peak in San Francisco; and working hard to have healthy relationships and bodies. Your parents seek to do everything with integrity from shopping for diapers to their Judaism. They may not be the most laid back parents but they will be tireless in their pursuit of your intellectual, spiritual and emotional health and happiness.

I cannot imagine a child in this world that was more wanted then you.

May you always feel surrounded by a family and community that loves and supports you.

May your connection to the Jewish people be a foundation and anchor in your life.

May you live a life of authenticity, having a strong self of sense wherever life takes you.

from your mom/ema

I don't know what a blessing is. When I look to the origins of the word, it's somewhat absurd. Will I give you your own land? A line like the stars? After this past week, I'm convinced no one should have that many children. But this line of inquiry is helpful. I can try to wish for you a journey like Abraham's, to describe your birthright as with Isaac, to convey what your name means as with Jacob. Your name in its entirety captures all of the life that we hope for you. Whimsy and uniqueness, strength. Adventure. Curiosity, as Milo learned in The Phantom Tollbooth. Real question asking and assumption turning. Abraham went on a journey lech lecha, into himself. May you know yourself well, may you be comfortable and make yourself uncomfortable. As you journey, may you be grounded in the comfort and security that you are loved as you are by many. May that security give you the confidence to go forth and explore with a genuinely open heart and kindness toward yourself and others.

You're being born into a pretty crazy world. Fifty years ago this week, three civil rights workers were killed in Mississippi for trying to make this truly a country of equality. Many things have improved in this world since then, but many have not. Your birthright is one of progressivism, of belief in a different world, one of social and now environmental justice, and dogged pursuit of that world. May you have courage to live your values. And may this life not make you crazy. "You are not obliged to finish the task, though neither are you free to desist from it."

Rav Kook wrote of an individual having a four fold song, of being aware of and attuned to his own soul's song, to that of his particular community, to that of humanity, and to that of the world. He says that in some, these songs "mix together at every moment and at all times." May you know yourself, and may the songs of your tradition, all humanity, and the world become part of your own song. May they stimulate you and may you contribute to them.

Your father and I spent a long time looking for each other, and then a long time looking for you. We got to really consider how wanted you are, how loved you would be. In this time, we have been fortunate to become immersed in a circle of friends and family who will love you and support you, stimulate and challenge you, make you laugh, and complement us in making sure you get what you need. They do all of that and more for us - we couldn't do this without them. They are our teachers, as friends and as parents. Many of them are here today and we're so grateful.

Your four grandparents, your aunt Moni and uncle Jeffrey, your great aunts and uncles, are simply pretty great. You will learn and gain attachment from them in ways we can't provide, thank goodness. They have given us so much, and we will have them in mind continually as the three of us find our way.

from your dad

Milo, may you give and receive much kindness. 

from everyone

Milo, as members of your community of family and friends,
We pledge this covenant of support to you and your parents
We promise to provide you with the warm comforts of community
To celebrate you in times of joy
To mourn with you in times of sorrow
To introduce you to new ideas and new perspectives
To offer you encouragement, acceptance and love.
We seal this covenant with this blessing:
Milo, be who you are
And may you be blessed in all that you are!

Introducing Milo Cousens Aufrecht

Beth Lynn Cousens and Joel S Aufrecht joyfully introduce to you Milo Cousens Aufrecht, born 16 June 2014 (18 Sivan 5774) and welcomed into his community in a Simchat Ben on 23 June 2014 (25 Sivan 5774).


Ruach avoteinu v’imoteinu, kayeim et hayeled hazeh l’aviv ul’imo. V’yikarei sh’mo b’yisrael Yosef Ben Basha Leah v’Yoel Shmuel. Yismach ha’av b’yotzei chalatzav v’tageil imo bifri bitna. Zeh hakaton gadol yih’ye. K’sheim sh’nichnas labrit, kein yikaneis latorah ul’ahava ul’ma’asim tovim.

May the spirit of our ancestors sustain this child through his parents’ loving care. Let him be known among our people Israel as Yosef, son of Basha Leah and Yoel Shmuel. May his mother and father rejoice in his creation and in the growth of his body and soul. May his name be a source of joy to him and may it inspire him to a life of Judaism, of learning, of love and of kindness. And let us say, Amen.



You are invited to read the thoughts and blessings of those who welcomed Milo on his eighth day.

Milo is preceded by several miscarriages and is the result of assisted reproduction. We are grateful for the many kindnesses and, perhaps more significantly, the medical talents of the doctors and researchers at UCSF Medical Center and Women's Health Clinic. We're not the first to go through this and unfortunately won't be the last. To that end, a call for action in Jewish communal life around support for ART is here and an outline of our experience is here. May they be healing to those who need them and may there be more babies in the world born to those who want them.

Tuesday, June 17, 2014

Beth Cousens and Joel Aufrecht are delighted to announce the birth of their child, on Monday afternoon, 16 June, 2014 (18 Sivan 5774). Mother and child are both well. A Simchat Ben ("Celebration of the Son") is planned in San Francisco for Monday, 23 June, with time and location to follow at http://aufrecht.org/yet. We are grateful for your kind support and expect to be ready for visitors by next week.

Finished the baby blanket; had a deadline

The baby blanket is finally done; I came under time pressure and had to focus.


The Honeycomb Brioche pattern was very tough: simply and easy enough to learn because it's just
1: k1, k1b
2: k2tog using dropped stitch, k1
3: k1b, k1
4: k1, k2tog using dropped stitch
BUT. It's extremely hard to undo mistakes, or even to spot them, and it's very easy to make mistakes, and with the 1x1 pattern, you can ruin a whole row very quickly. After a few false starts, I sort of had to grit my teeth and decide to try and fix only the worst mistakes, and to live with all the others, and once or twice I had to let even the worst mistakes go uncorrected. I was making up the whole pattern with the colored border (garter stitch, with a two-row and two-stitch garter stitch border in the blue) and had no idea how to join them, as you can see from the disastrous attempts at the bottom, and only learned the right way after it was too late to avoid some ugly patches. But it's done and I like it. Art/life metaphor, what?


The Ravelry project.


Saturday, June 14, 2014

Spain disproves the Moltke/Tyson Conjecture

No plan of operations extends with certainty beyond the first encounter with the enemy's main strength. 
— Field Marshal Helmuth von Moltke

Everybody has a plan until they get punched in the mouth.
— Michael Gerard Tyson

This is inexplicable. We trained all those weeks for this. The match has gone exactly as the coaching staff predicted.

Friday, June 13, 2014

Bonus Quiz Q: Kona reacts to the end of the first half of Cameroon vs Mexico


Kona is:

1) Happy that she's in a sunbeam while it rains in Natal
2) Disappointed by the superficiality of the the Univision announcers
3) Yawning

Monday, June 9, 2014

Another day, another app

This weekend I spent six hours trying to reproduce the tutorial application in Django. I guess it paid off, because this evening I was able to build and deploy my first real app on the new Aufrecht.org in about 2.5 hours, and maybe an hour of that was fussing with date equations, time zones, and date string formatting. I also discovered that time inside a repeatedly stopped and started virtual machine may not be reliable.

The app is something I thought of a few days ago in response to repeated inquiries. It's a "dead-man switch", which is used to change something for which, at the moment it must be changed, it is impossible to take action to change it.  So it's the opposite of a regular switch: it only activates when you stop pushing it.  The canonical example is an conductor's emergency brake on a train. Another example is this Outer Limits episode, which is so scary that even though I only saw bits and pieces of it while channel-surfing so long ago that I still had a TV, I'm never going to watch it again.  Here's another, even scarier example.

Since we have an event upcoming which fits this profile, in that I can update before it happens but probably not during, I coded up a dead man switch, which you can see here:

Aufrecht.org/yet

The Django tutorial app, or, Please, just show me a complete example

When I was evaluating different open-source web development platforms (which currently comes down Django vs Ruby on Rails, as far as I can tell), I tried making the tutorial app in Django.  The documentation is very thorough, but it's also a bit frustrating.  Part 1 is fine; you set up your new application, do some administrative paperwork, and create your data model.  Let's do that now; I want to recreate my "Songs not in 4/4 time" application.  The essential part is:
class Song(models.Model):
    TIME_CHOICES = (
        ('2/4', '2/4'),
        ('5/4', '5/4'),
        ('7/4', '7/4'),
        ('9/8', '9/8'),
        ('12/8', '12/8'),
        ('6/4', '6/4'),
        ('6/8', '6/8'),
        ('3/4', '3/4'),
        ('other', 'other')
    )

    artist = models.CharField(max_length=200)
    album = models.CharField(max_length=200)
    song = models.CharField(max_length=200)
    time = models.CharField(max_length=5,
                            choices=TIME_CHOICES)
I have a thing called a Song, and it has an artist, album, song, all text fields limited to 200 characters, and "time", which has to be one of a few common musical time signatures, such as 2/4, 3/4, or 6/8.

Okay, now to Part 2—wait a minute.

In 2000 and 2001 I worked for a startup called Westside that made ... cloud apps?  It's much harder to describe what they did than it seems like it should be, which might be part of why they aren't around any more, but: imagine Google doc spreadsheets, but with MS Access database structure.  That is, you describe your table (i.e., spreadsheet column headings), and you are done.  Say you are a plumber; this could be your appointment book.  Customer name, date of appointment, contact phone number, that's it.  Automatically, the website makes a form where you can type in new appointments, and a page where you can see all appointments, sorted or filter by any of the fields, paginated if it's too long.  Since date of appointment is a date, it can give you a calendar picker tool automatically.  All the basics of storing and changing data: create, replace, update, delete, query, all done without any code, just specifying the fields of your table.  And you can have multiple tables with relationships: e.g., list of Authors, list of Genres; each book has a Title but also an Author from the list and a Genre from the list.  No code needed.

It flopped for reasons that are obvious in retrospect: who's going to use it?  Small businesses don't want to do even the technical work of signing up to anything, much less thinking about their data model.  They want a full-featured application tailored to their business, and most of those features are not just storing, displaying, and modifying data in a set of tables.  They include credit card integration, pretty screens, advertising, posting to Facebook and Twitter, etc etc.  Big businesses can spend either nothing or way too much, so they are going to buy a million-dollar Enterprise-Grade solution from some consultants.  Web developers might have been a market, but it takes years to build an army of people using a particular technology, and years to make your technology good enough to deserve an army, and the startup had impatient investors.  It was still a much better idea than most things that get millions of dollars, but this was right at the trough between web boom one and web boom two, and that company is long gone even though it absolutely deserved (as much as anything else around) to get bought for a billion dollars.

My point is that that was over ten years ago but when I use the latest state-of-the-art web development platform, it hasn't caught up yet.  In Part 1 of the Django tutorial I defined my model; why is there a Part 2 at all?

It turns out that Django actually does do the same automatic interface creation that the Westside Webs did.  Part 2 walks you through a bit more "glue" code here and there, really fun stuff like:
from django.contrib import admin
from songs_not_4_4.models import Song

admin.site.register(Song) 
And then I have my list page:


And details, editing, creating, deleting:



But.  This whole automated shebang is strictly for administrators, and isn't designed to be exposed to the public as the actual interface to the data.  You have to write a bunch more code to get there.  So, back to the Django Tutorial, part 3.


To build the public-facing part of the website, you need lots more code.  And the tutorial walks you through all of the code.  The sample app, a "polls" app, is a bit more complicated than it needs to be to be a basic example.  And there's a really annoying thing in the tutorial: presenting lots of code that you shouldn't actually use.  I guess it's a pedagogical method, since this is a tutorial, to show you all of the impulsive initial solutions and what's wrong with them.  But there's way too much of that.  And, if you are just looking for an example of the correct approach to work from, this makes the tutorial a frustrating model.  If you go through the whole thing, here are all of the different versions of code you will write for just the "view" portion of the list screen.

from django.http import HttpResponse

def index(request):
    return HttpResponse("Hello, world. You're at the poll index.")
Okay, that's just a dummy response to reassure you that things are still working.  You want some real code:
from django.http import HttpResponse

from polls.models import Poll

def index(request):
    latest_poll_list = Poll.objects.order_by('-pub_date')[:5]
    output = ', '.join([p.question for p in latest_poll_list])
    return HttpResponse(output) 
But you don't really want to do it that way because "the page’s design is hard-coded in the view ... [instead,] use Django’s template system to separate the design from Python by creating a template that the view can use."  So do this instead:
from django.http import HttpResponse
from django.template import RequestContext, loader

from polls.models import Poll

def index(request):
    latest_poll_list = Poll.objects.order_by('-pub_date')[:5]
    template = loader.get_template('polls/index.html')
    context = RequestContext(request, {
        'latest_poll_list': latest_poll_list,
    })
    return HttpResponse(template.render(context))
Except that "[i]t’s a very common idiom to load a template, fill a context and return ... Django provides a shortcut."  So this, the third example, is the one you should actually follow:
from django.shortcuts import render

from polls.models import Poll

def index(request):
    latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
    context = {'latest_poll_list': latest_poll_list}
    return render(request, 'polls/index.html', context)
Okay, got it, use the shortcut.  Great.

Oh, by the way, Django has "generic views" that make automate the most common stuff.  So do this instead:
from django.views.generic import ListView
    from .models import Post

    class PostListing(ListView):
        model = Post
Would it have been so hard to say that in the first place?  Okay, so I want a reference model and the tutorial wants to tute.  But why couldn't they do both?
Happily, Daniel Samuels offers a complete model application with all sample code and no re-writes: https://github.com/danielsamuels/gcbv-article.

Thank you for your (dis)honesty, or epistomology of Proposition B

Looking for a place to park when driving to the Embarcadero, I got lost in the claims of the Embarcadero Center.  Waterfront development has been very controversial in San Francisco for decades, incorporating debates such as urban freeway construction and removal, public/private development of space and public access, legal limitations on development, etc etc.

Unlike the capitalization of lower-case letters on baseball jerseys ("Three primary schools of thought have emerged from these discussions: one that believes player names on jerseys should always be all caps, period; one that loves the lowercase letters; and a compromise position that advocates for the use of small caps instead of lowercase letters."), this is something that you actually need to have an opinion on, if you are a responsible citizen of the city, because we vote on this stuff fairly frequently.  And it can be hard to discern the truth, the best public policy, through the arguments and assertions.  Who has what agenda, and who can you trust?  Is an argument against waterfront development well-considered, or just greedy NIMBYism putting one group of rich people against another, and hurting the rest of the city?  Who can you trust?  So it's extremely helpful whenever a major developer gives you a clue about how to interpret their statements.  From the Embarcadero Center about page:
The Embarcadero Roadway Project has led to a complete renewal of the Downtown Waterfront District and ensures a bright future for Embarcadero Center. The Center is just steps away from the 42,000-seat AT&T Park, home to the San Francisco Giants baseball team.
Embarcadero Center to Phone Company Park is 1.3 miles walking, or over 3000 steps.  Okay, if they are lying that badly in the second sentence, how badly are they lying about "complete renewal" and "bright future"?  And who really plays baseball at the Park, and how many seats does it really have? (Okay, they aren't lying about the seat count.)

I voted for Prop B.  Of the many ways to improve the housing situation in San Francisco, keeping the path as smooth as possible for developers to get special treatment to build especially profitable luxury condos on the waterfront by violating existing planning rules isn't at the top of the list.

Friday, June 6, 2014

I love to debug stacks

I'm still chasing down why aufrecht.org takes over six seconds to load.  Since I have an Enterprise-Grade setup, it's an Enterprise-Grade hassle to debug.  Basically, the layers of stuff that happen to serve up a web page are:

  1. My browser
  2. The internet
  3. nginx
  4. Gunicorn
  5. python/django/django_cms
  6. the database
So the first step in debugging is to figure out which level the problem is happening at, with the caveat that, if the simplified model above is not appropriate to the actual problem, it will cause me to look in the wrong place for a very long time.

Apropos of that: I started debugging by noticing that, when I browsed to a page, it took six seconds for the nginx log to do anything.  Naturally, I suspected The Internet and wasted some time looking there.  But then I realized that I didn't know if nginx was logging when it first heard my request, or when it finished answering it.  I turned on detailed logging (a few changes to the config file and a restart), which gives me really good stuff like

2014/06/06 20:29:01 [debug] 7276#0: post event 00000000011555D0
2014/06/06 20:29:01 [debug] 7276#0: delete posted event 00000000011555D0
2014/06/06 20:29:01 [debug] 7276#0: accept on 0.0.0.0:80, ready: 0
2014/06/06 20:29:01 [debug] 7276#0: posix_memalign: 000000000110CFD0:256 @16
2014/06/06 20:29:01 [debug] 7276#0: *1091 accept: 50.174.84.3 fd:3
[300 more lines like this]
2014/06/06 20:29:02 [debug] 7276#0: *1091 writev: 374
2014/06/06 20:29:02 [debug] 7276#0: *1091 chain writer out: 0000000000000000
2014/06/06 20:29:02 [debug] 7276#0: *1091 event timer del: 8: 1402101002127
2014/06/06 20:29:02 [debug] 7276#0: *1091 event timer add: 8: 60000:1402101002127
2014/06/06 20:29:08 [debug] 7276#0: *1091 post event 00000000011556A0
2014/06/06 20:29:08 [debug] 7276#0: *1091 post event 0000000001168EB0
2014/06/06 20:29:08 [debug] 7276#0: *1091 delete posted event 0000000001168EB0
2014/06/06 20:29:08 [debug] 7276#0: *1091 http upstream request: "/foobar/?"
201
[almost 200 more lines]
2014/06/06 20:29:08 [debug] 7276#0: *1091 malloc: 0000000001115530:1024
2014/06/06 20:29:08 [debug] 7276#0: *1091 recv: fd:3 -1 of 1024
2014/06/06 20:29:08 [debug] 7276#0: *1091 recv() not ready (11: Resource temporarily unavailable)
2014/06/06 20:29:08 [debug] 7276#0: *1091 free: 0000000001115530


I've trimmed a lot to show you the exciting part.  Notice anything about the timestamps?

Yes, there's the six-second gap.  The nginx debug log responds almost immediately to the request, then waits for something for six seconds, then replies.  So I can rule out The Internet (almost certainly) and nginx (probably) as locations for the delay.  Now I have to figure out what gunicorn is doing.

That actually turns out to be equally simple.  I turn on debugging and restart it, and I can see that the log responds only a second after I make a request.  Okay, so it's Django or Python or Django-cms that's taking six seconds.  Let's turn on the next debug log and look.

Um.  Okay, it doesn't seem like this is going to be simple.  There's lots of instructions on how to write some complicated stuff to the configuration that ... does nothing more than enable you to then add your own debugging commands within your own code.  Ack.  This is not going to be easy.  I am going to need Enterprise-Grade chocolate.

UPDATE an hour later (because this is so boring I'm not going to make another post altogether for this UPDATE): when I disable memcache, the page loads in about one second.  Good enough for now; making memcache help instead of hurt goes on the backlog for later.

Deploying my new web site: One step forward, six seconds back.

I finished setting up an Industrial-Strength Django website on my current favorite hosting site, Digital Ocean, and said what the heck, let's go ahead and make it the real aufrecht.org.  Since my blog (that you are reading right now) is actually just Google Blogger, hosted at blog.aufrecht.org, and my email is Gmail, the only thing on aufrecht.org is a static home page placeholder:


So I edited the home page of the new Django-CMS site, and aside from the detail that I couldn't get Kona's pictured centered even though I clicked Center in the CMS, and the detail that in order to change the "djangocms" logotype to "aufrecht.org" I had to learn what a png sprite was (Its existence is driven by the same detail that affects grocery store line optimization: the overhead of downloading a new file to a web browser is like the overhead of somebody fumbling for their checkbook right at the end, not only having not bothered to bag their own damned groceries but having chosen to spend that entire time staring into space with a faint smirk rather than digging out their checkbook.  Three people with three items each take much more time than one person with nine items, provided the checker isn't an arthritic person forced into post-retirement manual labor by the insatiable greed of the 0.01%.  Taking the parallel back to web development, obviously you are better off with one big graphic file than fifty tiny ones, because the Walton family's tax-dodging fake charities make your network slow.) and then I had to force myself to just manually edit the sprite in a temporary hack that will disappear as soon as I regenerate static content rather than spend 2 more hours learning the best and most Pythonic way to override Django's defaults—a css shim, editing the original package, a hack in Ruby?, but after that, my new Digital Ocean server, temporarily set up as pre.aufrecht.org (with separate database server) was ready to become aufrecht.org. 


So I went to my DNS provider to make the switch but I couldn't remember which company is hosting the current static page and I tried to check and discovered that, somehow, between with 1and1's confusing interface and my shaky command of the exact details of DNS lingo, I apparently already made the switch yesterday.  I guess that when I tried to set the IP address for pre.aufrecht.org to the server, and I set the "IP Address" and not the "CNAME" (can you set a CNAME to an IP?  In keeping with the general obstinacy of this project, I'm not even going to look that up right now, but I am going to create this link which I will click on later), and instead of setting pre.aufrecht.org, that changed the whole aufrecht.org IP to point to my new site.  So, setting up a new Digital Ocean server pair from scratch using my new instructions took 4.3 hours, and making that site be aufrecht.org took either 55 minutes or negative one day, depending on how you count, so I may have come out ahead by about 19 hours in this whole thing.  I'm still not sure exactly what happened with the CNAME IP thing, and did it mean that for most of a day aufrecht.org was coming back with an nginx proxy error, and who was actually hosting the previous aufrecht.org anyway, and do I still owe them money ... but this captures my feelings:
But if Future Wolverine from the original Dark Future wakes up in the head of Future Wolverine from the Bright Future, doesn’t that mean that there is a separate Wolverine Consciousness who suddenly gets wiped out of existence when the Original Wolverine Consciousness comes back to the Bright Future?
Well-
But if in the original timeline Mystique was captured by Trask’s people, why didn’t she spend the original trilogy trying to get back at the people who experimented on her?
I mean, there could be an explana-
And in the original timeline, how did Magneto ...
Anyway.  Everything on aufrecht.org looks almost the same as it did before, but instead of a simple HTML page and image and web server, there is now an entire Enterprise-grade infrastructure of nginx, supervisor running gunicorn, django-cms, memcached, and a host-separated, ssl- and private-network-secured PostgreSQL server providing the same thing.  And Kona is now left justified.  And the page takes seven seconds to load.  What?

Webpagetest says 6.858 seconds on first view, .5 seconds on repeat.


Gtmetrix says 6.94 seconds.


Pingdom can do it in 6.18 seconds


And in all cases, the web page itself is pretty fast, milliseconds, but somehow the page request sits for six seconds before it even hits the webserver, as verified by watching the logs while browsing the site.  It's true that I picked Digital Ocean's Amsterdam server farm out of a vague and not-really-thought-out protest of the NSA's illegal and anti-democratic domestic spying program, but even if the servers are all smoking pot it shouldn't take six seconds.

In a waterfall process I might say that was a bug and spend all weekend trying to figure it out.  But in my Agile process, where I'm both the Team Member and the Product Owner, I am declaring 6 seconds acceptable for the original task of setting up aufrecht.org, and getting that down to under a second is going to be a new task, to be done later.  Maybe by Kona, who in the year or two since the temporary page was created has not found any new snoozing positions:




Pop Quiz

Which is best?

a) To save up to fifteen percent on car insurance
b) Chips and a sub for only 5.25
c) To crush your enemies, to see them driven before you, and to hear the lamentations of their women

Wednesday, June 4, 2014

Industrial-strength Web Development coming through, watch your toes

Progress to report:

I have a working, more-or-less-professional-grade Django-CMS website working in a virtual machine on my desktop computer.  That took over six hours (not all at once).  And, I have it running with the database on a separate (virtual) computer.  And I duplicated that setup so that I have two complete, independent copies of the pair of computers, one for development and one for production.  And I got everything into Mercurial, and made a change on development, and put it in a central repository, and took that change from the central repository to the production site, and it (eventually) worked.  That took about 8 hours, of which at least three hours was setting up PostgreSQL to accept remote connections secured by SSL certificates.  I'll tell you about that some day.

Here's what it looks like:


And here's what it looks like from behind (these are the consoles for the four virtual computers running in VirtualBox):


Progress usually looks boring, though, right?

Sunday, June 1, 2014

WTF is Middleware, or, "... the full complexities of the problem ...", or, why knowing "HTML" is not enough to build websites

I'm getting deeper into setting up a modern production open-source webserver, still sort-of following the instructions I like, by George London. However, since that guide is built around Amazon Web Services, and I think that Amazon is not kosher, it's not as simple for me as just following the guide literally.  I'm still (still!) somewhat stuck on figuring out what exactly I want to do.  I want to have a production website with all my blog writing, my songs not in 4/4 database, and my experimental task tracking tool; and I want to get current with state-of-the-art website development. But I still have "Step 2: ????" in the middle of my plan.

Let's focus on setting up web servers. There are a lot of tricks and problems to running a website. The last platform I used, OpenACS, already had a good grasp on many of them in 2001, so I want to see what's really changed in 13 years. The key problems are:
  1. Changes to a website need to be tested before they are put in production
  2. A website usually includes both code and data, so changes to code need to include changes to data structures
  3. Your website is too slow/broken.
There are many solutions to each of these problems.  They basically involve massive quantities of Middleware.  Middleware is the stuff that is completely essential but that anybody not in software has never heard of much less conceived of the need for.

For testing, the basic idea is that changes are made in one environment and then deployed to a production environment when ready.  At a minimum, this includes knowing a version control tool; I know cvs quite well but everybody is praising the new paradigm of distributed version control (git or Mercurial); I started with git but I think Mercurial is a better fit for my needs.  This also entails having multiple environments, and there are two ways to have multiple environments: manually setting up and configuring each server (aka, the hard way), or learning to use an tool that can automatically set up and configure each server (aka, the even harder way that claims to lead to an easy way).  Fab, as I understand it, help with this, but on top of that George London's approach uses chef. I think I'm going to proceed with manual configuration for now, since I need to understand that anyway before automating, and then I may try chef or SaltStack or something else.

For managing changes to code and corresponding data structures properly, something that I still see plenty of developers treat as a surprising and unexpected new problem, Django seems to be committing to south, so I'll use that.

For the third thing, web sites being too slow and broken, things have gotten much more complicated.  Content Delivery Networks, CDNs, are far more common and reaching, I think, a bit lower than they were in 2001.  Load balancing (spreading work over multiple webservers and multiple database servers) used to mean buying an expensive box from BigIP, but at least at the lower end there seem to be a lot more software solutions (nginx in this case).  But the biggest change is a much stronger emphasis on caching.  That is, after computing what some part of a website should be like, remembering that result and skipping the computation until the result is no longer valid.  Just for that problem alone, the standard Django approach seems to include memcached, celery, and RabbitMQ.

And with all that, we still haven't gotten to the actual web site code, the part that determines if you have created Twitter or just pets.com.  That code needs a home to run, since most modern web code is written in interpreted languages, not compiled languages; programs written in compiled languages are turned into freestanding programs before use, but interpreted code is always run within an interpreter program.  Django is Python code, so it runs in Python, but for production purposes that should happen within something more industrial-strength, and the suggested option is Gunicorn.  And I've already decided to use (or at least get much further before giving up on) Django itself, which is a web development platform, and Django-cms, which is a module for Django that does articles and directories and page templates and the like (like Wordpress or Blogger, but more and bigger).

So, to get from the default development Django environment, which is basically just two programs,
- Python
- SQLite

to a full production environment, I need to understand at least the basics of:

- PostgreSQL
- Nginx
- memcached
- gunicorn
- celery
- RabbitMQ
- Mercurial
- fabric
- south

Each of these has the potential to be weird, buggy, idiosyncratic, complicated to install, tough to maintain, abruptly abandoned by its creators, or any number of other bad things.  As of today I'm experienced and comfortable with exactly one of these, PostgreSQL.  So step 1 is to set up a production-ready server inside a virtual server on my own desktop, using all of these tools, adding them one at a time.  Today, I got PostGreSQL working, and nginx, but at that point my static files weren't getting served, and diagnosing that started to suggest that celery and RabbitMQ need to go in at the same time as nginx, and celery has friends, celerycam and celerybeat, and dj-celery.  So I started digging into the documentation, where I found this very reassuring line:

[Celery] is easy to use so that you can get started without learning the full complexities of the problem it solves.
I guess that's the crux of it all, right?  Since each of these ten or twenty or thirty helper programs represents years of work and decades (hopefully) of knowledge of the problem, it's really nice when they just work and solve problems you didn't even know you had.

And so far, they aren't terrible.  The really demoralizing thing in middleware work is to discover that the tool you have committed to is broken in a subtle way, or grossly inadequately documented, or simply doesn't do the thing it's purported to do.  Microsoft's Team Foundation Server does most of the stuff by itself that these dozens of programs collectively do, but for the parts that I've had to use (admittedly mostly the non-development side, like SharePoint, the wiki, and other tools) are pretty awful in every regard, from performance to documentation to feature set to style.  So from that perspective, I guess this project is going well.