This is a technique that I discovered several years ago while teaching an introductory programming course, and I was extremely pleased to find out that although it had already been discovered, I had given it the same name. Programming by wishful thinking is a style of top-down program design that helps you improve the structure of your code, isolate the hairy parts from the simple parts, think more clearly about data flow, and work faster. I find it particularly helpful for novice programmers who feel intimidated when starting a new project and aren’t sure how to get started, but I still practise it regularly as well.
So how does it work? I think this might be easiest to demonstrate by example. Let’s imagine that you are trying to write a Python program that checks the xkcd RSS feed and downloads new comics. In a bottom-up design, we can think about the different built-in functions of Python and its standard library that we could use. We’ll need to make a web request to the RSS feed, parse XML, download images, sounds pretty complicated. Wouldn’t it be great if Python had a few more built-ins, and we could just do this?
items = fetch_rss('https://www.xkcd.com/rss.xml') for item in items: src = get_img_from_item(item) download_image(src)
We’d be done! Although Python is a batteries
included language, it unfortunately doesn’t include these
functions. That’s okay though, because we can just define each of
these functions. Let’s start with
fetch_rss. This is
definitely going to involve a web request, so we should figure out
which library to use. Then we’ll need an XML parser to convert the
response into a list of items. Sounds pretty complicated, and I
really just want to download some images. Can we come back to this
For now, let’s just fake it. At the time of publication, the xkcd RSS
feed contains the following four comics:
Models of the Atom,
Missal of Silos,
Magnetic Pole, and
Thor Tools. So we know that if
we were to actually implement
fetch_rss and run it right
now, those are the results we’d get.
def fetch_rss(url): return [ 'https://imgs.xkcd.com/comics/models_of_the_atom.png', 'https://imgs.xkcd.com/comics/missal_of_silos.png', 'https://imgs.xkcd.com/comics/magnetic_pole.png', 'https://imgs.xkcd.com/comics/thor_tools.png' ]
There we go. Good enough for now. Next up,
get_img_from_item. This is supposed to extract from one
of the items in the result of
fetch_rss the URL of the
image to download. Well that’s easy, because each item in the result
already is that URL.
def get_img_from_item(item): return item
The last function to implement is
actually care about this one working properly, because I actually
want to view the images on my harddrive. We don’t have much choice
but to implement it properly. We can use
urllib.request.urlretrieve for this, but we need to
specify the target filename. Let’s just use more wishful thinking and
from urllib.request import urlretrieve def download_image(src): dst = generate_filename(src) urlretrieve(src, dst)
I said the previous function was the last function, but we’ve shifted the goalposts now by introducing yet another imaginary function. No problem, we can implement it too. For now, we’ll just stick the download into the working directory using whatever filename the source URL uses (get the last element in the path). Maybe in the future we’ll want something a bit better than this, but since we’ve tucked it away in its own function it’s completely isolated and we can easily come back and change it.
def generate_filename(url): return url.split('/')[-1]
Put it all together in a single file (in the right order), run it, and sure enough I have four comics to view locally. From Make it Work, Make it Right, Make it Fast, we’ve made it work, now it’s time to make it right by going back and properly implementing the functions we skipped over.
from urllib.request import urlopen from xml.etree import ElementTree def fetch_rss(url): src = urlopen(url).read() xml = ElementTree.fromstring(src) return xml.findall('channel/item')
By correctly implementing
fetch_rss, we have changed the
return value and thereby broken
get_img_from_item, so we
should implement it now too.
def get_img_from_item(item): desc = item.find('description').text html = ElementTree.fromstring(desc) return html.get('src')
And now we’ve made it right. The approach we have just followed pretty accurately reflects the actual process I use when developing software. Here’s why.
- Getting to a working solution as quickly as possible is hugely motivating.
- Helps me to think about the structure of the code first, and worry about implementation details later.
- Forces me to think about data flow early, by identifying the required parameters and return values of each function, and the links between successive function calls.
- Helps to isolate the hairier parts of the task in well-specified functions, decreasing coupling with the main program body and making it easier to change or improve the implementations later without breaking things.
- The hard-coded implementations can easily be converted to unit tests.