The Python Book

Python essentials

1 More imports from Tkinter import * Poker Dice Code Listing

We’ve added the new imported global dice
modules we need to make Tkinter

work and keep the rest the same 01 from ttk import * dice1_check = dice1.get()
import random dice2_check = dice2.get()

2 Dice list from itertools import groupby dice3_check = dice3.get()

The list that holds the dice is kept 02 dice = 0 07 dice4_check = dice4.get()
outside the main function so that it
can be accessed everywhere def roll(roll_number): dice5_check = dice5.get()
dice_rerolls = [dice1_check,

3 Rolls 03 numbers = range(1,7) dice2_check, dice3_check, dice4_check,
dice = range(roll_number) dice5_check]
Same goes for the roll function. iterations = 0
It doesn’t specifically need to be while iterations < roll_number: for i in range(len(dice_rerolls)):
inside the gui function anyway if 0 in dice_rerolls:
iterations = iterations + 1 dice_rerolls.remove(0)

4 Decisions dice[iterations-1] = random. 08 if len(dice_rerolls) == 0:
result = “You finish with “ +
The checkboxes in the graphical choice(numbers)
code we’re going to create later will
return dice hand(dice)

give us numbers we can analyse for hand_output.set(result)

the code. We retrieve these numbers def hand(dice): else:

and check them to find out which dice_hand = [len(list(group)) for key, roll_number = len(dice_rerolls)

dice the user wishes to re-roll group in groupby(dice)] number_rerolls = roll(roll_num-

5 Hands dice_hand.sort(reverse=True) ber)
straight1 = [1,2,3,4,5]
dice_changes = range(len(dice_
Finally, our hand analysis function straight2 = [2,3,4,5,6]
rerolls))
is the last part of the original code if dice == straight1 or dice ==
iterations = 0
that is kept outside the gui function. straight2:
while iterations < roll_number:
Both this and the above function return “a straight!”
pass the necessary details back elif dice_hand[0] == 5: iterations = iterations + 1
up the chain to then be added into
the new graphical elements of the return “five of a kind!” dice_changes[iterations-1]
new interface elif dice_hand[0] == 4:
= number_rerolls[iterations-1]
return “four of a kind!”
04 iterations = 0

6 No dice elif dice_hand[0] == 3: 09 while iterations < roll_number:
if dice_hand[1] == 2: iterations = iterations + 1
If no dice have been selected to return “a full house!”
re-roll, the hand output is changed replacement = number_

to show a final message else: rerolls[iterations-1]

return “three of a kind.” dice[dice_

7 Re-roll elif dice_hand[0] == 2: changes[iterations-1]] = replacement

This part is almost the same as if dice_hand[1] == 2: dice.sort()

before – a new set of dice are rolled return “two pair.” new_dice_list = [0,0,0,0,0]

and then inserted into the list of dice else: for i in range(len(dice)):

like before, then re-sorted to make return “one pair.” new_dice_list[i] =

the hand analysis easier else: names[dice[i]]

return “a high card.” final_dice = “ “.join(new_dice_

8 More functions list)

The new gui function is the main def gui(): dice_output.set(final_dice)
change to the Poker Dice code, global dice
and as before includes the Tkinter dice = roll(5) final_result = “You finish with
elements and other parts of the dice.sort()
original code nine = 1 10 “ + hand(dice)

hand_output.set(final_result)

9 Game start ten = 2 def reset_game():
A simple function that we can use to 05 jack = 3 global dice

activate the re-rolls of the dice queen = 4 dice = roll(5)

king = 5 dice.sort()
for i in range(len(dice)):
10 New hand ace = 6 11
empty_dice[i] = names[dice[i]]
The new dice are named, analysed, names = { nine: “9”, ten: “10”, jack: first_dice = “ “.join(empty_dice)
and everything is then set for the gui dice_output.set(first_dice)
to display the final outcome “J”, queen: “Q”, king: “K”, ace: “A” }

result = “You have “ + hand(dice)

11 Reset 06 def game(): result = “You have “ + hand(dice)
throws() hand_output.set(result)
Like with the hangman code, we
have a function to reset all the def throws(): if __name__ == ‘__main__’:
variables, allowing you to start the gui()
game again

The Python Book 51

Python essentials

#!/usr/bin/env python2 accessed at all points in the code. Python 2 does
not allow you to call upon global variables when
01 from Tkinter import * you’re in a nested function, whereas in Python 3
this could have gone into the gui function.
from ttk import *
from random import * 03 Graphical function
word = 0 We’re putting all the working code into
the gui function so it can be activated from the
02 word_length = 0 main interface. This means we can import the
Hangman code into the interface without the
clue = 0 game window popping up, and only run it when
we activate the gui function from here.
03 def gui():
04 Random word
global word, word_length, clue We bring in the three variables with
global so we can modify them throughout
dictionary = [“gnu”,”kernel”,”linux”,”mageia”,”penguin”,”ubuntu”] the code, and then set the word. As before, a
random item from the list of words is selected
word = choice(dictionary) YOU LOSE with choice, the length is ascertained, and the
clue to display is set.
04 word_length = len(word)
05 The hanged man
clue = word_length * [“_”] The main difference this time for the
Hangman graphics is that instead of printing
tries = 6 When you’ve run out of guesses, these out, we’re going to display them in the
interface. When the function is called and the
def hangedman(hangman): the game stops. From here, you graphic selected, it’s placed in the variable we’ve
graphic = [ can also reset the game to play set up in the interface code that we’re using to
“”” again if you wish. display the result.

+-------+ 06 Games begin
All the analysis of the letter we’ve
|| entered is done in this function. To that end, we
start by obtaining the incorrect guesses so far
05 |O from the variable we’ve set up so the interface
| -|- can access it if we want it to. The letter from
the entry field in the interface is then obtained
| /\ and cleaned up so it can be used with the rest of
the code.
|
07 Check the letter
=============== This section of the code is again largely
unchanged – the letter is taken and compared to
“””] the word with find to see if it matches with one
of the letters. The if statement then adds one to
graphic_set = graphic[hangman] the incorrect guess variable, or updates the clue
variable to add the letter in the right spot.
hm_graphic.set(graphic_set)
08 Update interface
def game(): These three lines set the graphic for this
round, join the current clue together as a string,
06 letters_wrong = incorrect_guesses.get() and then set it on the variable for the interface
to read.
letter=guess_letter()
09 Update scores
first_index=word.find(letter) Exactly as before, we check to see if the
player has won or lost yet. In the event of either,
if first_index == -1: a message is displayed to signify this, and the
wins and losses score is updated using set.
letters_wrong +=1

07 incorrect_guesses.set(letters_wrong)

else:

for i in range(word_length):

if letter == word[i]:

clue[i] = letter

hangedman(letters_wrong)

08 clue_set = “ “.join(clue)

word_output.set(clue_set)

if letters_wrong == tries:

result_text = “Game Over. The word was “ + word

result_set.set(result_text)

new_score = computer_score.get()

new_score += 1

09 computer_score.set(new_score)
if “”.join(clue) == word:

result_text = “You Win! The word was “ + word

result_set.set(result_text)

new_score = player_score.get()

new_score += 1

player_score.set(new_score)

01 First lines ttk, for the grid code we’ll be using to align the
As usual, we start off each program with different elements.
the code that lets us run it in the command line,
followed by importing the necessary modules: 02 Global variables
random, to determine the word to use; Tkinter, We have kept these three variables
for the majority of the graphical code; and outside of the gui function so they can be

52 The Python Book

Python essentials

def guess_letter(): ORIGINAL INTERFACE THE HANGMAN GUI
letter = letter_guess.get()
You’ll also need the interface Press the updated Hangman
10 letter.strip() code from last issue, which button to launch a new window.
letter.lower() already works with the modified Here we have the initial graphic,
return letter Rock, Paper, Scissors code. The word clue and entry for the player
way it was left off means it won’t to interact with. The scores
def reset_game(): work with the new code, so you’ll are set to zero, and no result
global word, word_length, clue have to change the command in message is displayed as no
incorrect_guesses.set(0) each button from [game].start games have been played yet.
hangedman(0) to [game].gui.
result_set.set(“”)
letter_guess.set(“”)
word = choice(dictionary)
word_length = len(word)
clue = word_length * [“_”]
new_clue = “ “.join(clue)
word_output.set(new_clue)

11 hm_window = Toplevel()
hm_window.title (“Hangman”)
incorrect_guesses = IntVar()
incorrect_guesses.set(0)
player_score = IntVar()

12 computer_score = IntVar()
result_set = StringVar()
letter_guess = StringVar()
word_output = StringVar()
hm_graphic = StringVar()

hm_frame = Frame(hm_window, padding = ‘3 3 12 12’, width = 300)

13 hm_frame.grid(column=0, row = 0, sticky=(N,W,E,S))
hm_frame.columnconfigure(0, weight=1)
hm_frame.rowconfigure(0,weight=1)

Label(hm_frame, textvariable = hm_graphic).grid(column=2, row = 1)
Label(hm_frame, text=’Word’).grid(column=2, row = 2)

14 Label(hm_frame, textvariable = word_output).grid(column=2, row = 3)

Label(hm_frame, text=’Enter a letter’).grid(column=2, row = 4)

15 hm_entry = Entry(hm_frame, exportselection = 0, textvariable = letter_guess).grid(column = 2, row = 5)
hm_entry_button = Button(hm_frame, text = “Guess”, command = game).grid(column = 2, row = 6)

Label(hm_frame, text = “Wins”).grid(column = 1, row = 7, sticky = W)
Label(hm_frame, textvariable = player_score).grid(column = 1, row = 8, sticky = W)
Label(hm_frame, text = “Losses”).grid(column = 3, row = 7, sticky = W)

16 Label(hm_frame, textvariable = computer_score).grid(column = 3, row = 8, sticky = W)
Label(hm_frame, textvariable = result_set).grid(column = 2, row = 9)
replay_button = Button(hm_frame, text = “Reset”, command = reset_game).grid(column = 2, row = 10)

if __name__ == ‘__main__’:
gui()

10 Sanitise input The frame is set 15 Text entry
The guess_letter function purely gets up as before Entry here sets a text box we will add the
the letter from the player input variable, strips it letters to. The exportselection option makes it
of any formatting, makes it lower case, and then 13 Framed window so selecting the letter won’t immediately copy it
returns it back to the game function. This is so The frame is set up the same way as to the clipboard, and the textvariable selection
the letter can be used properly. last time. We pad the frame from the edge of is where the code stores the letter added. The
the window, set a grid, give it sticky points at button activates the game function, analysing
11 New window compass points, and allow for setting objects the letter the player entered.
We use the Toplevel command from with specific row and column points.
Tkinter like last month to separate the loops of 16 Results and reset
the main interface and game window. We then 14 Clue to Hangman The rest of the code is similar to what
use title to call it Hangman. These labels are fairly straightforward we’ve done already: labels to display fixed text
– we’re either giving them fixed text, or telling and the scores/result text that change. The
12 Interface variables them to use a specific textvariable so they can button that activates the reset function is also
Tkinter only works with specific variables be updated as we play the game. put at the bottom here. The final two lines allow
– we’ve created all the ones we need or can use us to import the module into the interface code.
here. IntVars take integers, while StringVars take
strings. We’ve used get and set throughout the
rest of the code with these to get and set values.

The Python Book 53

Python essentials

#!/usr/bin/env python2 EXTRA GAME FUNCTIONS

17 Start over 17 from Tkinter import * We mentioned that the game function
from ttk import * doesn’t necessarily need to be used right
The usual array of command-line now. You can either clean up the code and
import random
compatibility and module importing here. The
from itertools import groupby

groupby function is specifically imported here 18 dice = 0 remove it, or add extra functions, such
as being able to choose a random new
for dice analysis.

18 Outside dice def roll(roll_number): selection of dice, or making it two-player.
For Poker Dice, there’s only one variable numbers = range(1,7) Experiment with what you want to do!
dice = range(roll_number)

to show at any one time, the dice. Again, due to 19 iterations = 0
the nested functions, and because we’re using while iterations < roll_number:
Python 2, we need to call it with global from here
iterations = iterations + 1
dice[iterations-1] = random.choice(numbers)

to make sure the game can be reset properly. return dice

19 Dice rolls def hand(dice):
The roll function has been removed from dice_hand = [len(list(group)) for key, group in groupby(dice)]
the gui function so as not to create any code dice_hand.sort(reverse=True)
errors with some of its variables. It can be easily straight1 = [1,2,3,4,5]
called within the nested functions. It hasn’t straight2 = [2,3,4,5,6]
if dice == straight1 or dice == straight2:

changed at all from the original code. 20 return “a straight!” THE POKER DICE GUI
elif dice_hand[0] == 5:
20 Hand of dice Two things are being printed out
Like roll, nothing has changed for the return “five of a kind!” on the initial window. The first
hand function. It’s simply now placed outside elif dice_hand[0] == 4: set of dice, ordered in the way
the gui function for the exact same reasons. we did last time, and the current
It also means that you can easily import this return “four of a kind!” hand. The checkboxes activate
function into another script if you wish. elif dice_hand[0] == 3: a specific number that is used
when re-rolling dice with the
21 GUI start if dice_hand[1] == 2: Reroll button.
As we’ve mentioned last month and in return “a full house!”
the Hangman code, we put all the GUI code into
a function so that we can call on it when we want else:
to. In this case, pressing the Poker Dice button return “three of a kind.”

elif dice_hand[0] == 2:
if dice_hand[1] == 2:
return “two pair.”
else:
return “one pair.”

else:

on the main interface activates pokerdice.gui, return “a high card.”

which is this function. 21 def gui():

22 First roll global dice
As the window opens, we immediately dice = roll(5)
make the first roll. This is then sorted, each dice.sort()
number is attributed to a card, and then the nine = 1
ten = 2

result is created to be displayed in the main 22 jack = 3
window. This is similar to how it worked before, queen = 4

but instead it’s now entered into the StringVars king = 5
for the interface towards the end of the script ace = 6
names = { nine: “9”, ten: “10”, jack: “J”, queen: “Q”, king: “K”,

23 Start game ace: “A” }
When we activate the button that starts result = “You have “ + hand(dice)

game, it immediately sends us to the rest of the 23 def game():
code. This would also work if you had the button throws()

go to the throws function instead; however, you def throws():
can add other functions to this part if you wish. global dice

24 Dice selection 24 dice1_check = dice1.get()
The first thing we do is find out what dice2_check = dice2.get()
checkboxes have been ticked by the player. We dice3_check = dice3.get()
then put these in a list so we can change out the dice4_check = dice4.get()
dice5_check = dice5.get()

correct dice numbers. We’ve also brought in dice dice_rerolls = [dice1_check, dice2_check, dice3_check, dice4_
so we can check against that what the current check, dice5_check]

dice rolls are.

54 The Python Book

Python essentials

25 for i in range(len(dice_rerolls)): The check
26 if 0 in dice_rerolls: buttons are new
27 dice_rerolls.remove(0)
25 Dice to re-roll
28 if len(dice_rerolls) == 0: If a checkbox isn’t selected, we have
result = “You finish with “ + hand(dice) it set to give a zero value. We want to remove
hand_output.set(result) these from the list so that the correct dice
are changed, so we use the for loop to check
else: each part of the list, and then use the remove
roll_number = len(dice_rerolls) function when the element does equal zero.
number_rerolls = roll(roll_number)
dice_changes = range(len(dice_rerolls)) 26 Early finish
iterations = 0 If no dice have been selected to re-roll,
while iterations < roll_number: the list will contain all 0s, which will then be
iterations = iterations + 1 removed. The length of this list will then also be
dice_changes[iterations-1] = number_rerolls[iterations-1] zero, meaning we can use that to end the game if
iterations = 0 the player hits Reroll without selecting any dice.
while iterations < roll_number:
iterations = iterations + 1 27 New dice
replacement = number_rerolls[iterations-1] This else function works roughly
dice[dice_changes[iterations-1]] = replacement the same as before. We start by getting the
dice.sort() necessary information for how many dice to roll,
new_dice_list = [0,0,0,0,0] and a list to put the re-rolls. We then roll as many
for i in range(len(dice)): new dice as we need with the first while loop
new_dice_list[i] = names[dice[i]]
final_dice = “ “.join(new_dice_list) 28 Game over
dice_output.set(final_dice) We use the same kind of while loop to
final_result = “You finish with “ + hand(dice) replace the new numbers into the original list,
hand_output.set(final_result) much like last time. Then the dice are re-sorted,
analysed, joined as a string and then set into the
def reset_game(): ONE WINDOW interface’s variable. The final hand message is
global dice also create and set.
dice = roll(5) The way we’ve made these Tkinter
dice.sort() interfaces is to have the games 29 Graphical variables
for i in range(len(dice)): launch in a separate window. You As we’re rolling the dice as soon as
empty_dice[i] = names[dice[i]] can have them all running in one we launch the game, but the interface code
first_dice = “ “.join(empty_dice) window, though, by replacing the doesn’t start until the end, you can see that
dice_output.set(first_dice) labels and buttons of the original after creating the necessary variables, we also
result = “You have “ + hand(dice) interface by putting them as then set them. Of note, the dice have to be made
hand_output.set(result) different functions or classes. into a string separately with the for loop before
Make sure to add a quit button to adding to the variable.
pd_window = Toplevel() the games that lets you go back
pd_window.title (“Poker Dice”) to the main page. 30 Check buttons
dice_output = StringVar() The main new addition to this code is
empty_dice = [0,0,0,0,0] the check buttons with Checkbutton. You can
for i in range(len(dice)): set an on and off value, with default off being 0.
We’ve made it so that the check buttons return
empty_dice[i] = names[dice[i]] the same number as the dice they’re changing,
first_dice = “ “.join(empty_dice) which we explained how we used earlier in the
dice_output.set(first_dice) code. The variable option sets whatever the
outcome is to the specific Tkinter variable.
29 hand_output = StringVar()
hand_output.set(result)
dice1 = IntVar()
dice2 = IntVar()
dice3 = IntVar()
dice4 = IntVar()
dice5 = IntVar()
result_set = StringVar()
player_score = IntVar()
computer_score = IntVar()

pd_frame = Frame(pd_window, padding = ‘3 3 12 12’, width = 300)
pd_frame.grid(column=0, row = 0, sticky=(N,W,E,S))
pd_frame.columnconfigure(0, weight=1)
pd_frame.rowconfigure(0,weight=1)
Label(pd_frame, text=’Dice’).grid(column=3, row = 1)
Label(pd_frame, textvariable = dice_output).grid(column=3, row = 2)
Label(pd_frame, textvariable = hand_output).grid(column=3, row = 3)

Label(pd_frame, text=’Dice to Reroll?’).grid(column=3, row = 4)
reroll1 = Checkbutton(pd_frame, text = “1”, variable = dice1, onvalue = 1, offvalue
= 0).grid(column=1, row = 5)
reroll2 = Checkbutton(pd_frame, text = “2”, variable = dice2, onvalue = 2, offvalue
= 0).grid(column=2, row = 5)

30 reroll3 = Checkbutton(pd_frame, text = “3”, variable = dice3, onvalue = 3, offvalue
= 0).grid(column=3, row = 5)
reroll4 = Checkbutton(pd_frame, text = “4”, variable = dice4, onvalue = 4, offvalue
= 0).grid(column=4, row = 5)
reroll5 = Checkbutton(pd_frame, text = “5”, variable = dice5, onvalue = 5, offvalue
= 0).grid(column=5, row = 5)
pd_reroll_button = Button(pd_frame, text = “Reroll”, command = game).grid(column =
3, row = 6)
replay_button = Button(pd_frame, text = “Reset”, command = reset_game).grid(column
= 3, row = 7)

if __name__ == ‘__main__’:
gui()

The Python Book 55

Python essentials

Build an app for
Android with Python

Master Kivy, the excellent cross-platform application
framework to make your first Android app…

Q Here we've drawn all the simple graphics
for our game… now we just have to make
the shapes actually do something!

The great thing about Kivy is there are loads style game. We'll then be able to compile this Once you've mastered Kivy, your imagination
of directions we could take it in to do some straight to an Android APK that you can use just is the only limit. If you're pretty new to Kivy,
pretty fancy things. But, we're going to make like any other. don't worry, we won't assume that you have
a beeline for one of Kivy's coolest features any pre-existing knowledge. As long as you
- the ability it affords you to easily run your Of course, once you have mastered the have mastered some of the Python tutorials
programs on Android. basic techniques you aren't limited to using in this book so far, and so have a fairly good
any particular kind of app, as even on Android understanding of the language, you shouldn’t
We'll approach this by first showing how to you can make use of all your favourite Python have any problems following along.
make a new app, this time a dynamic Breakout- libraries to make any sort of program you like.

Before anything else, let's throw together a to be drawn anywhere on your screen and on its children in proportion to its own position
basic Kivy app (Fig. 01). We've pre-imported any widget type. and size – so no matter where we run it or
the widget types we'll be using, which this how we resize the window, it will place all the
time are just three: the basic Widget with Before we can do any of this we'll need a class game objects appropriately.
no special behaviour, the ModalView with a for each kind of game object, which we’re going
pop-up behaviour as used last time, and the to pre-populate with some of the properties Next we can use Kivy's graphics instructions
FloatLayout as we will explaine later. Kivy that we'll need later to control them. Remember to draw various shapes on our widgets. We'll
has many other pre-built widgets for creating from last time, Kivy properties are special just demonstrate simple rectangles to show
GUIs, but this time we’re going to focus on attributes declared at class level, which (among their locations, though there are many more
drawing the whole GUI from scratch using other things) can be modified via kv language advanced options you might like to investigate.
Kivy's graphics instructions. These comprise and dispatch events when they are modified In a Python file we can apply any instruction
either vertex instructions to create shapes (Fig. 02). by declaring it on the canvas of any widget, an
(including rectangles, lines, meshes, and so example of which is shown in Fig. 03.
on) or contextual graphics changes (such as The Game class will be one big widget
translation, rotation, scaling, etc), and are able containing the entire game. We've specifically This would draw a red rectangle with the
made it a subclass of FloatLayout because same position and size as the player at its
this special layout is able to position and size moment of instantiation – but this has a

56 The Python Book

Python essentials

problem, unfortunately, as the drawing is Once you have the basic techniques,
static. When we later move the player widget, you aren’t limited to one app… your
the red rectangle will stay in the same place, imagination is the only limit
and the widget will be invisible when it is in its
real position. from kivy.app import App Fig. 01
from kivy.uix.widget import Widget
We could fix this by keeping references to our from kivy.uix.floatlayout import FloatLayout
canvas instructions and repeatedly updating from kivy.uix.modalview import ModalView
their properties to track the player, but there's
actually an easier way to do all of this - we __version__ = '0.1' # Used later during Android compilation
can use the Kivy language we introduced last
time. It has a special syntax for drawing on class BreakoutApp(App):
the widget canvas, which we can use to draw pass
each of our widget shapes:
BreakoutApp().run()
<Player>:
canvas: from kivy.properties import (ListProperty, NumericProperty, Fig. 02
Color: ObjectProperty, StringProperty)
rgba: 1, 1, 1, 1
Rectangle: class Game(FloatLayout): # Will contain everything
pos: self.pos blocks = ListProperty([])
size: self.size player = ObjectProperty() # The game's Player instance
ball = ObjectProperty() # The game's Ball instance
<Ball>:
canvas: class Player(Widget): # A moving paddle
Color: position = NumericProperty(0.5)
rgb: 1, 0.55, 0 direction = StringProperty('none')
Rectangle:
pos: self.pos class Ball(Widget): # A bouncing ball
size: self.size # pos_hints are for proportional positioning, see below
pos_hint_x = NumericProperty(0.5)
<Block>: pos_hint_y = NumericProperty(0.3)
canvas: proper_size = NumericProperty(0.)
Color: velocity = ListProperty([0.1, 0.5])
rgb: self.colour
# A property we predefined above class Block(Widget): # Each coloured block to destroy
Rectangle: colour = ListProperty([1, 0, 0])
pos: self.pos
size: self.size from kivy.graphics.context_instructions import Color Fig. 03
Color: from kivy.graphics.vertex_instructions import Rectangle
rgb: 0.1, 0.1, 0.1
Line: class Player(Widget):
rectangle: def __init__(self, **kwargs):
[self.x, self.y, super(Player, self).__init__(**kwargs)
self.width, self.height] with self.canvas:
Color(1, 0, 0, 1) # r, g, b, a -> red
The canvas declaration is special, underneath Rectangle(pos=self.pos, size=self.size)
it we can write any canvas instructions we # or without the with syntax, self.canvas.add(...)
like. Don't get confused, canvas is not a
widget and nor are graphics instructions
like Line. This is just a special syntax that is
unique to the canvas. Instructions all have
different properties that can be set, like the
pos and size of the rectangle, and you can
check the Kivy documentation online for all
the possibilities. The biggest advantage is
that although we still declare simple canvas
instructions, kv language is able to detect
what Kivy properties we have referred to and
automatically track them, so when they are
updated (the widget moves or is resized) the
canvas instructions move to follow!

The Python Book 57

Python essentials

Q Running the app shows our coloured blocks on the <Game>:
screen… but they all overlap! We can fix that easily ball: the_ball
player: the_player
You probably noticed we had one of the this special property, and it is used by Ball:
Block’s ‘Color’ instructions refer to its colour FloatLayouts like our Game to set their id: the_ball
property. This means that we can change position proportionate to the layout. Player:
the property any time to update the colour id: the_player
of the block, or in this case to give each block The dictionary is able to handle
a random colour (Fig. 04). various parameters, but in this You can run the game again now, and should
case ‘x’and ‘y’ give x and y Block be able to see all the graphics working
Now that each of our widgets has a graphical position as a relative fraction of the properly. Nothing moves yet, but thanks to
representation, let’s now tell our Game where to parent width and height. the FloatLayout everything should remain in
place them, so that we can start up the app and proportion if you resize the game/window.
actually see something there. You can run the app now, and this time
it will add 50 blocks to the Game before Now we just have to add the game
class Game(FloatLayout): displaying it on the screen. Each should have mechanics. For a game like this you usually
def setup_blocks(self): one of the three possible random colours want to run some update function many times
for y_jump in range(5): and be positioned in a grid, but you'll now per second, updating the widget positions and
for x_jump in range(10): notice their sizes haven't been manually set carrying out game logic – in this case collisions
block = Block(pos_hint={ so they all overlap. We can fix this by setting with the ball (Fig. 06).
'x': 0.05 + 0.09*x_jump, their size_hint properties – and let's also
'y': 0.05 + 0.09*y_jump}) take this opportunity to do the same for the The Clock can schedule any function at
self.blocks.append(block) other widgets as well (Fig. 05). any time, either once or repeatedly. A function
self.add_widget(block) scheduled at interval automatically receives the
This takes care of keeping all our game time since its last call (dt here), which we've passed
class BreakoutApp(App): widgets positioned and sized in proportion through to the ball and player via the references
def build(self): to the Game containing them. Notice that we created in kv language. It's good practice to
g = Game() the Player and Ball use references to the scale the update (eg ball distance moved) by this
g.setup_blocks() properties we set earlier, so we'll be able to dt, so things remain stable even if something
return g move them by just setting these properties interrupts the clock and updates don't meet
and letting kv language automatically update the regular 1/60s you want.
Here we create the widgets we want then use their positions.
add_widget to add them to the graphics tree. Our At this point we have also added the first steps
root widget on the screen is an instance of Game The Ball also uses an extra property to toward handling keyboard input, by binding to
and every block is added to that to be displayed. remain square rather than rectangular, just the kivy Window to call a method of the Player
because the alternative would likely look a every time a key is pressed. We can then finish
The only new thing is that every Block little bit odd. off the Player class by adding this key handler along
has been given a pos_hint. All widgets have with touch/mouse input.
We've now almost finished the basic
graphics of our app! All that remains is to add class Player(Widget):
a Ball and a Player widget to the Game. def on_touch_down(self, touch):
self.direction = (
'right' if touch.x > self.parent.

center_x else 'left')

def on_touch_up(self, touch):
self.direction = 'none'

def on_key_down(self, keypress,
scancode, *args):

if scancode == 275:
self.direction = 'right'

elif scancode == 276:
self.direction = 'left'

else:
self.direction = 'none'

def on_key_up(self, *args):
self.direction = 'none'

def update(self, dt):
dir_dict = {'right': 1, 'left': -1,

58 The Python Book

Python essentials

'none': 0} import random Fig. 04
self.position += (0.5 * dt * dir_ Fig. 05
class Block(Widget):
dict[self.direction]) def __init__(self, **kwargs): Fig. 06
super(Block, self).__init__(**kwargs)
These on_touch_ functions are Kivy's general self.colour = random.choice([
method for interacting with touch or mouse input, (0.78, 0.28, 0), )0.28, 0.63, 0.28), )0.25, 0.28, 0.78)])
they are automatically called when the input
is detected and you can do anything you like in <Block>:
response to the touches you receive. In this case size_hint: 0.09, 0.05
we set the Player's direction property in response # ... canvas part
to either keyboard and touch/mouse input, and
use this direction to move the Player when its <Player>:
update method is called. We can also add the right size_hint: 0.1, 0.025
behaviourfortheball(Fig. 07). pos_hint: {'x': self.position, 'y': 0.1}
# ... canvas part
This makes the ball bounce off every wall by
forcing its velocity to point back into the Game, <Ball>:
as well as bouncing from the player paddle – pos_hint: {'x': self.pos_hint_x, 'y': self.pos_hint_y}
but with an extra kick just to let the ball speed size_hint: None, None
change. It doesn't yet handle any interaction proper_size:
with the blocks or any win/lose conditions, min(0.03*self.parent.height, 0.03*self.parent.width)
but it does try to call Game.lose() if the size: self.proper_size, self.proper_size
ball hits the bottom of the player's screen, # ... canvas part
so let's now add in some game end code to handle
all of this (Fig. 08). And then add the code in Fig. 09 from kivy.clock import Clock
to your 'breakout.kv 'file. from kivy.core.window import Window
from kivy.utils import platform
This should fully handle the loss or win,
opening a pop-up with an appropriate message class Game(FloatLayout):
and providing a button to try again. Finally, we def update(self, dt):
have to handle destroying blocks when the self.ball.update(dt) # Not defined yet
ballhitsthem(Fig. 10). self.player.update(dt) # Not defined yet

This fully covers these last conditions, checking def start(self, *args):
collision via Kivy's built-in collide_widget method Clock.schedule_interval(self.update, 1./60.)
that compares their bounding boxes (pos and
size). The bounce direction will depend on how far def stop(self):
the ball has penetrated, as this will tell us how it Clock.unschedule(self.update)
first collided with the Block.
def reset(self):
So there we have it, you can run the code to for block in self.blocks:
play your simple Breakout game. Obviously it's self.remove_widget(block)
very simple right now, but hopefully you can self.blocks = []
see lots of different ways to add whatever extra self.setup_blocks()
behaviour you like – you could add different self.ball.velocity = [random.random(), 0.5]
types of blocks and power-ups, a lives system, self.player.position = 0.5
more sophisticated paddle/ball interaction, or
even build a full game interface with a menu and class BreakoutApp(App):
settings screen as well. def build(self):
g = Game()
We’re just going to finish showing one cool thing if platform() != 'android':
that you can already do – compile your game for Window.bind(on_key_down=g.player.on_key_down)
Android! Generally speaking you can take any Kivy Window.bind(on_key_up=g.player.on_key_up)
app and turn it straight into an Android APK that g.reset()
will run on any of your Android devices. You can Clock.schedule_once(g.start, 0)
even access the normal Android API to access return g
hardware or OS features such as vibration,
sensors or native notifications.

We'll build for Android using the Buildozer tool,
and a Kivy sister project wrapping other build
tools to create packages on different systems.
This takes care of downloading and running the
Android build tools (SDK, NDK, etc) and Kivy's
Python-for-Android tools that create the APK.

The Python Book 59

Python essentials

class Ball(Widget) Fig. 07 Here you will be needing some basic
def update(self, dt): Fig. 08 dependencies, which can be installed with
self.pos_hint_x += self.velocity[0] * dt Fig. 09 ease just by using your distro's normal
self.pos_hint_y += self.velocity[1] * dt repositories. The main ones to use are
if self.right > self.parent.right: # Bounce from right OpenJDK7, zlib, an up-to-date Cython,
self.velocity[0] = -1 * abs(self.velocity[0]) and Git. If you are using a 64-bit distro you will also
if self.x < self.parent.x: # Bounce from left be in need of 32-bit compatibility libraries for zlib,
self.velocity[0] = abs(self.velocity[0]) libstdc++, as well as libgcc. You can then go on and
if self.top > self.parent.top: # Bounce from top download and install Buildozer:
self.velocity[1] = -1 * abs(self.velocity[1])
if self.y < self.parent.y: # Lose at bottom git clone git://github.com/kivy/buildozer
self.parent.lose() # Not implemented yet cd buildozer
self.bounce_from_player(self.parent.player) sudo python2.7 setup.py install

def bounce_from_player(self, player): When you’re done with that part you
if self.collide_widget(player): can then go on and navigate to your
self.velocity[1] = abs(self.velocity[1]) Kivy app, and you’ll have to name the main code file
self.velocity[0] += ( ‘main.py’, this is the access point that the Android
0.1 * ((self.center_x - player.center_x) / APK will expect. Then:
player.width))
buildozer init
class GameEndPopup(ModalView):
message = StringProperty() This creates a ‘buildozer.spec’ file, a settings file
game = ObjectProperty() containing all the information that Buildozer needs
to create your APK, from the name and version to
class Game(Widget): the specific Android build options. We suggest that
def lose(self): you check through the whole file just to see what's
self.stop() available but most of the default settings will be
GameEndPopup(message='[color=#ff0000]You lose![/color]', fine,theonlythingwesuggestchangingis(Fig. 11).
game=self).open()
There are various other options you will often
def win(self): # Not called yet, but we'll need it later want to set, but none are really all that vital right
self.stop() now, so you’re able to immediately tell Buildozer to
GameEndPopup(message='[color=#00ff00]You win![/color]', build your APK and get going!
game=self).open()
buildozer android debug
<GameEndPopup>:
size_hint: 0.8, 0.8 This will take some time, so be patient and it will
auto_dismiss: False # Don't close if player clicks outside work out fine. When you first run it, it will download
BoxLayout: both the Android SDK and NDK, which are large
orientation: 'vertical' (at least hundreds of megabytes) but vital to the
Label: build. It will also take time to build these and to
text: root.message compile the Python components of your APK. A lot
font_size: 60 of this only needs to be done once, as future builds
markup: True will take a couple of minutes if you change the
halign: 'center' buildozer.spec, or just a few seconds if you've only
Button: changed your code.
size_hint_y: None
height: sp(80) The APK produced is a debug APK, and you can
text: 'Play again?' install and use it but there are extra steps if you
font_size: 60 want to fully digitally sign it so that it can be posted
on_release: root.game.start(); root.dismiss() on the Play store. This isn't hard, and Buildozer
can do some of the work, but you can check the
documentation online for full details.

Assuming everything goes fine (it should!),
your Android APK will be in a newly created 'bin'
directory with the name ‘KivyBreakout-0.1-debug.
apk’. You can send it to your phone any way you
like (eg email), though you may need to enable
application installation from unknown sources in
your Settings before you can install it.

60 The Python Book

Python essentials

Putting your APK self.parent.do_layout() Fig. 10
on the Play Store self.parent.destroy_blocks(self) Fig. 11

Find out how to digitally sign a release class Game(FloatLayout):
APK and upload it to an app store of def destroy_blocks(self, ball):
your choice for i, block in enumerate(self.blocks):
if ball.collide_widget(block):
1Build and sign a release APK y_overlap = (
First we have to begin by creating a personal ball.top - block.y if ball.velocity[1] > 0
digital key, then using it to digitally sign a else block.top - ball.y) / block.size_hint_y
special release version of the APK. Run these x_overlap = (
commands, and follow the instructions they then ball.right - block.x if ball.velocity[0] > 0
give you. else block.right - ball.x) / block.size_hint_x
if x_overlap < y_overlap:
## Create your personal digital key ball.velocity[0] *= -1
## You can choose your own else:
## keystore name, alias, and passwords. ball.velocity[1] *= -1
$ keytool -genkey -v -keystore test-
release-key.keystore \ self.remove_widget(block)
self.blocks.pop(i)
-alias test-alias -keyalg RSA if len(self.blocks) == 0:

-keysize 2048 -validity 10000 self.win()
## Compile your app in release mode return # Only remove at most 1 block per frame
$ buildozer android release
## Sign the APK with your new key title = Kivy Breakout # Displayed in your app drawer
$ jarsigner -verbose -sigalg package.name = breakout # Just a unique identifying string,
SHA1withRSA -digestalg SHA1 \
# along with the package.domain
-keystore ./test-release-key.keystore \ fullscreen = 0 # This will mean the navbar is not covered
./bin/KivyBreakout-0.1-release- log_level = 2 # Not vital, but this will print a lot more debug
unsigned.apk test-alias
## Align the APK zip file # information and may be useful if something
$ ~/.buildozer/android/platform/android- # goes wrong
sdk-21/tools/zipalign -v 4 \
./bin/KivyBreakout-0.1-release-
unsigned.apk \
./bin/KivyBreakout-0.1-release.apk

2Sign up as a Google
Play Developer
Visit https://play.google.com/
apps/publish/signup, and follow
the instructions. You'll need to pay a
one-off $25 charge, but then you can
upload as many apps as you like.

3Upload your app to the store
Click 'Add new application'
to submit your app the store,
including uploading your APK and
adding description text. When
everything is ready, simply click
Publish, and it should take just a few
hours for your app to go live!

Q Your game should run on any modern Android device… you
can even build a release version and publish to an app store!

Python essentials

Making web apps with Python

Python provides quick and easy way to build
applications, including web apps. Read on to find out
how to use it to build a feature-complete web app

Python is known for its simplicity and plain old CGI modules to utilising fully groomed
capabilities. At this point it is so advanced web frameworks. Using the latter is the most
that there is nothing you cannot do with popular method of building web applications
Python, and conquering the web is one of the with Python, since it allows you to build
possibilities. When you are using Python for web applications without worrying about all that
development you get access to a huge catalogue low-level implementation stuff. There are many
of modules and community support – make the web frameworks available for Python, such
most of them. as Django, TurboGears and Web2Py. For this
tutorial we will be using our current preferred
Web development in Python can be done option, Django.
in many different ways, right from using the

Resources 01 Creating the Django Project 02 Configuring the Django project
magazine issue tracker settings
Python 2.7: The django-admin.py file is used to create new Before we start working on the application,
Django projects. Let’s create one for our issue let’s configure the Django project as per our
https://www.python.org/download/releases/2.7/ tracker project… requirements.

Django version 1.4: In Django, a project represents the site and Edit ludIssueTracker/settings.py as follows
its settings. An application, on the other hand, (only parts requiring modification are shown):
https://www.djangoproject.com/ represents a specific feature of the site, like
blogging or tagging. The benefit of this approach Database Settings: We will be using SQLite3
is that your Django application becomes as our database system.
portable and can be integrated with other NOTE: Red text indicates new code or
Django sites with very little effort. updated code.
$ django-admin.py startproject ‘default’: {
ludIssueTracker
‘ENGINE’: ‘django.
A project directory will be created. This will also db.backends.sqlite3’,
act as the root of your development web server
that comes with Django. Under the project ‘NAME’: ‘ludsite.db3,
directory you will find the following items…
manage.py: Python script to work with your Path settings
project.
ludIssueTracker: A python package (a directory Django requires an absolute path for directory
with __init__.py file) for your project. This settings. But we want to be able to pass in the
package contains your project’s settings and relative directory references. In order to do that
configuration data. we will add a helper Python function. Insert the
ludIssueTracker/settings.py: This file contains following code at the top of the settings.py file:
all the configuration options for the project. import os
ludIssueTracker/urls.py: This file contains def getabspath(*x):
various URL mappings.
wsgi.py: An entry-point for WSGI-compatible return os.path.join(os.path.
web servers to serve your project. Only useful abspath(os.path.dirname(__file__)),
when you are deploying your project. For this *x)
tutorial we won’t be needing it. Now you can update the path options:
@code
TEMPLATE_DIRS = (

getabspath(‘templates’)
)
MEDIA_ROOT = getabspath(‘media’)

62 The Python Book

Python essentials

MEDIA_URL = ‘/media/’ ('new', 'New'), You just installed Django's auth
('accepted','Accepted'), system, which means you don't have
Now we will need to enable the admin interface ('reviewed','Reviewed'), any superusers defined.
for our Django site. This is a neat feature of Django ('started','Started'), Would you like to create one now?
which allows the automatic creation of an admin ('closed','Closed'), (yes/no): yes
interface of the site based on the data model. The )
admin interface can be used to add and manage 05 Enabling the admin site
content for a Django site. class Issue(models.Model): The admin site is already enabled,
# owner will be a foreign key but we need to enable it in the urls.py file – this
Uncomment the following line: contains the regex-based URL mapping from
INSTALLED_APPS = ( to the User model which is already model to view. Update the urls.py file as follows:
built-in Django from django.conf.urls import
‘django.contrib.auth’, patterns, include, url
‘django.contrib.contenttypes’, owner = models.ForeignKey(User,n from django.contrib import admin
‘django.contrib.sessions’, ull=True,blank=True) admin.autodiscover()
‘django.contrib.sites’,
‘django.contrib.messages’, # multichoice with defaulting to urlpatterns = patterns(‘’,
‘django.contrib.staticfiles’, "new" url(r’^admin/’, include(admin.
‘django.contrib.admin’,
# ‘django.contrib.admindocs’, status = models.CharField(max_ site.urls)),
) length=25,choices=ISSUE_STATUS_ )
CHOICES,default='new')
03 Creating ludissues app 06 Starting the Django web server
In this step we will create the primary summary = models.TextField() Django includes a built-in web server
app for our site, called ludissues. To do that, we # date time field which will be which is very handy to debug and test Django
will use the manage.py script: set to the date time when the record applications. Let’s start it to see how our admin
$ python manage.py startapp is created interface works…
ludissues opened_on = models.
DateTimeField('date opened', auto_ To start the web server:
We will need to enable this app in the config file now_add=True) $ python manage.py runserver
as well: modified_on = models.
INSTALLED_APPS = ( DateTimeField('date modified', auto_ If you do not have any errors in your code, the
now=True) server should be available on port 8000. To
............. launch the admin interface, navigate your
'django.contrib.admin', def name(self): browser to http://localhost:8000/admin.
‘ludissues’, return self.summary.
) You will be asked to log in here. Enter the
split('\n',1)[0] username and password that you created while
04 Creating the data model syncing the database.
This is the part where we define the # Admin front end for the app. We
data model for our app. Please see the inline are also configuring some of the Q Admin login screen
comments to understand what is happening. # built in attributes for the admin
From django.db import models: interface on After logging in, you will notice that all the apps
# We are importing the user # how to display the list, how it installed in your project are available here. We are
authentication module so that we use will be sorted only interested in the Auth and LudIssues app.
the built # what are the search fields etc.
# in authentication model in this class IssueAdmin(admin.ModelAdmin): You can click the +Add to add a record. Click
app the Add button next to Users and add a few
from django.contrib.auth.models date_hierarchy = 'opened_on' users to the site.
import User list_filter = ('status','owner')
# We would also create an admin list_display = ('id','name','sta Once you have the users inside the system,
interface for our app tus','owner','modified_on') you can now add a few issues to the system.
from django.contrib import admin search_fields =
['description','status']
# A Tuple to hold the multi choice
char fields. # register our site with the Django
# First represents the field name admin interface
the second one repersents the admin.site.
display name register(Issue,IssueAdmin)
ISSUE_STATUS_CHOICES = (
To have the created data model reflected in the
database, run the following command:
$ python manage.py syncdb
You’ll be also asked to create a superuser for it:

The Python Book 63

Python essentials

Q Admin homepage 07 Creating the public user interface template names
Click the Add button next to Issues. Here you for ludissues #which will be looked in the default
At this point, the admin interface is working. But template
will notice that you can enter Owner, Status we need a way to display the data that we have #directories
and Summary for the issue. But what about added using the admin interface. But there is no
the opened_on and modified_on field that public interface. Let’s create it now. url(r’^$’,’object_
we defined while modelling the app? They list’,info,name=’issue-list’),
are not here because they are not supposed We will have to begin by editing the main
to be entered by the user. opened_on will urls.py (ludIssueTracker/urls.py). url(r’^(?P<object_
automatically set to the date time it is created urlpatterns = patterns(‘’, id>\d+)/$’,’object_
and modified_on will automatically set to the detail’,info,name=’issue-detail’),
date time on which an issue is modified. (r’^’,include(‘ludissues.
urls’)), )
Another cool thing is that the owner field is
automatically populated with all the users inside (r’^admin/’, include(admin.site. To display an issue list and details, we are using
the site. urls)), a Django feature called generic views. In this
) case we are using views called list and details.
We have defined our list view to show ID, This allow us to create an issue list view and
name, status, owner and ‘modified on’ in the This ensures that all the requests will be issue detail view. These views are then applied
model. You can get to this view by navigating to processed by ludissues.urls first. using the issue_list.html and issue_detail.html
http://localhost:8000/admin/ludissues/issue/. template. In the following steps we will create
08 Creating ludissues.url the template files.
Q The ‘Add issue’ menu Create a urls.py file in the app directory
(ludissues/urls.py) with the following content: 09 Setting up template and media
from django.conf.urls import directories
patterns, include, url In this step we will create the template and
# use ludissues model media directories. We have already mentioned
from models import ludissues the template directory as
TEMPLATE_DIRS = (
# dictionary with all the objects in
ludissues getabspath(‘templates’)
info = { )

‘queryset’:ludissues.objects. Which translates to ludIssueTracker/
all(), ludIssueTracker/templates/. Since we will be
} accessing the templates from the ludissues
app, the complete directory path would be
# To save us writing lots of python ludIssueTracker/ludIssueTracker/templates/
code ludissues. Create these folders in your
# we are using the list_detail project folder.
generic view
Also, create the directory ludIssueTracker/
#list detail is the name of view we ludIssueTracker/media/ for holding the CSS
are using file. Copy the style.css file from the resources
urlpatterns = patterns(‘django. directory of the code folder.
views.generic.list_detail’,
#issue-list and issue-detail are the To serve files from this folder we need to make
it available publicly. To do that, open settings.py
and add the following lines in ludIssueTracker/
ludIssueTracker/urls.py:
from django.conf.urls import
patterns, include, url
from django.conf import settings
# Uncomment the next two lines to
enable the admin:
from django.contrib import admin
admin.autodiscover()

Q The list view for issues urlpatterns = patterns(‘’,
(r’^’,include(‘ludissues.

urls’)),
(r’^admin/’, include(admin.site.

64 The Python Book

Python essentials

urls)),
(r’^media/

(?P<path>.*)$’,’django.views.static.
serve’,

{‘document_root’:settings.
MEDIA_ROOT})
)

10 Creating the template files Q The magazine Issue Tracker in action – list of issues
Templates will be loaded from the
ludIssueTracker/ludIssueTracker/templates Now we need to create the issue_list.html ludIssueTracker/ludIssueTracker/templates/
directory. In Django, we start with the template. This template is responsible for ludissues/issue_detail.html
ludIssueTracker/ludIssueTracker/templates/ displaying all the issues available in the system. {% extends ‘base.html’ %}
base.html template. Think of it as the master ludIssueTracker/ludIssueTracker/templates/ {% block title %}Issue #{{ object.id
template which can be inherited by slave ones. ludissues/issue_list.html }} - {% endblock %}
ludIssueTracker/ludIssueTracker/templates/ {% extends ‘base.html’ %} {% block content %}
base.html {% block title %}View Issues - {% <h2>Issue #{{ object.id }} <span>{{
<!DOCTYPE html PUBLIC “-//W3C//DTD endblock %} object.status }}</span></h2>
XHTML Strict//EN” {% block content %} <div class=”issue”>
“ HYPERLINK “http://www.w3.org/ <table cellspacing=”0”
TR/xhtml1/DTD/xhtml1-strict.dtd” class=”column-options”> <h2>Information</h2>
http://www.w3.org/TR/xhtml1/DTD/ <div class=”date”>
xhtml1-strict.dtd”> <tr>
<html> <th>Issue</th> <p class=”cr”>Opened {{
<th>Description</th> object.opened_on }} ago</p>
<head> <th>Status</th>
<title>{% block title %}{% <th>Owner</th> <p class=”up”>Last modified
{{ object.modified_on }} ago</p>
endblock %}LUD Issues</title> </tr>
<link rel=”stylesheet” {% for issue in object_list %} </div>
<tr> <div class=”clear”>&nbsp;</div>
href=”{{ MEDIA_URL }}style.css” <div class=”block w49 right”>
type=”text/css” media=”screen” /> <td><a href=”{% url issue-
detail issue.id %}”>{{ issue.id }}</ <p class=”ass title”>Owner</
</head> a></td> p>
<body>
<td><a href=”{% url issue- <p class=”ass”>{{ object.
<div id=”hd”> detail issue.id %}”>{{ issue.name owner }}</p>
<h1>LUD Issue }}</a></td>
</div>
Tracker</span></h1> <td>{{ issue.status }}</td> <div class=”clear”>&nbsp;</div>
</div> <td>{{ issue.owner}}</td> <div class=”block”>
<div id=”mn”> </tr>
<ul> {% endfor %} <p class=”des
<li><a </table> title”>Summary</p>
{% endblock %}
href=”{% url issue-list %}” <p class=”des”>{{ object.
class=”sel”>View Issues</a></li> Here we are inheriting the base.html file that we summary }}</p>
created earlier. {% for issue in object_list %}
<li><a runs on the object sent by the urls.py. Then we </div>
href=”/admin/”>Admin Site</a></li> are iterating on the object_list for issue.id and </div>
issue.name. {% endblock %}
</ul>
</div> Now we will create issue_detail.html. This And that’s everything! The issue tracker app is
<div id=”bd”> template is responsible for displaying the detail now complete and ready to use. You can now
view of a case. point your browser at localhost:8000 to start
{% block content %} using the app.
{% endblock %}

</div>
</body>
</html>

{{ variablename }} represents a Django variable.
(% block title %} represents blocks. Contents
of a block are evaluated by Django and are
displayed. These blocks can be replaced by the
child templates.

The Python Book 65

Python essentials

50Pythtiposn
Python is a programming language that lets you work more quickly and
integrate your systems more effectively. Today, Python is one of the most
popular programming languages in the open source space. Look around
and you will find it running everywhere, from various configuration tools
to XML parsing. Here is the collection of 50 gems to make your Python
experience worthwhile…

Basics command at the command prompt (>>>), one by It is also important to remember that Python
one, and the answer is immediate. takes tabs very seriously – so if you are
1. Running Python scripts Python interpreter can be started by issuing receiving any error that mentions tabs, correct
the command: the tab spacing.
On most of the UNIX systems, you can run $ python
Python scripts from the command line. kunal@ubuntu:~$ python 3. Dynamic typing
$ python mypyprog.py Python 2.6.2 (release26-maint, Apr
19 2009, 01:56:41) In Java, C++, and other statically typed
2. Running Python [GCC 4.3.3] on linux2 languages, you must specify the data type of
programs from Type “help”, “copyright”, “credits” the function return value and each function
Python interpreter or “license” for more information. argument. On the other hand, Python is
>>> <type commands here> a dynamically typed language. In Python
The Python interactive interpreter makes it In this article, all the code starting at the you never have to explicitly specify the data
easy to try your first steps in programming and >>> symbol is meant to be given at the type of anything. Based on what value you
using all Python commands. You just issue each Python prompt. assign, Python will keep track of the data
type internally.

66 The Python Book

Python essentials

4. Python statements x,y = my_function.minmax(25, 6.3) dateobj = DateTime(string)

Python uses carriage returns to separate 9. Module defined names 14. Converting a list
statements, and a colon and indentation to to a string for display
separate code blocks. Most of the compiled Example:
programming languages, such as C and C++, use The built-in function ‘dir()’ can be used to find You can convert a list to string in either of the
semicolons to separate statements and curly out which names a module defines. It returns a following ways.
brackets to separate code blocks. sorted list of strings. 1st method:
>>> import time >>> mylist = [‘spam’, ‘ham’, ‘eggs’]
5. == and = operators >>> dir(time) >>> print ‘, ‘.join(mylist)
[‘__doc__’, ‘__file__’, ‘__name__’, spam, ham, eggs
Python uses ‘==’ for comparison and ‘=’ for ‘__package__’, ‘accept2dyear’, 2nd method:
assignment. Python does not support inline ‘altzone’, ‘asctime’, ‘clock’, >>> print ‘\n’.join(mylist)
assignment, so there’s no chance of accidentally ‘ctime’, ‘daylight’, ‘gmtime’, spam
assigning the value when you actually want to ‘localtime’, ‘mktime’, ‘sleep’, ham
compare it. ‘strftime’, ‘strptime’, ‘struct_ eggs
time’, ‘time’, ‘timezone’, ‘tzname’,
6. Concatenating strings ‘tzset’] 15. Tab completion
in Python interpreter
You can use ‘+’ to concatenate strings. 10. Module internal
>>> print ‘kun’+’al’ documentation You can achieve auto completion inside Python
kunal interpreter by adding these lines to your .pythonrc
You can see the internal documentation (if file (or your file for Python to read on startup):
7. The __init__ method available) of a module name by looking at import rlcompleter, readline
.__doc__. readline.parse_and_bind(‘tab: complete’)
The __init__ method is run as soon as Example: This will make Python complete partially typed
an object of a class is instantiated. The >>> import time function, method and variable names when you
method is useful to do any initialization >>> print time.clock.__doc__ press the Tab key.
you want to do with your object. The clock() -> floating point number
__init__ method is analogous to a constructor in This example returns the CPU time or real time 16. Python
C++, C# or Java. since the start of the process or since the first documentation tool
Example: call to clock(). This has as much precision as the
class Person: system records. You can pop up a graphical interface for searching
the Python documentation using the command:
def __init__(self, name): 11. Passing arguments $ pydoc -g
self.name = name to a Python script You will need python-tk package for this to work.

def sayHi(self): Python lets you access whatever you have passed 17. Python
print ‘Hello, my name is’, self.name to a script while calling it. The ‘command line’ documentation server
content is stored in the sys.argv list.
p = Person(‘Kunal’) import sys You can start an HTTP server on the given port on
p.sayHi() print sys.argv the local machine. This will give you a nice-looking
Output: access to all Python documentation, including
[~/src/python $:] python initmethod.py 12. Loading modules or third-party module documentation.
Hello, my name is Kunal commands at startup $ pydoc -p <portNumber>

8. Modules You can load predefined modules or 18. Python development
commands at the startup of any Python software
To keep your programs manageable as they script by using the environment variable
grow in size, you may want to break them up into $PYTHONSTARTUP. You can set environment There are plenty of tools to help with Python
several files. Python allows you to put multiple variable $PYTHONSTARTUP to a file which development. Here are a few important ones:
function definitions into a file and use them as a contains the instructions load necessary IDLE: The Python built-in IDE, with
module that can be imported into other scripts and modules or commands . autocompletion, function signature popup help,
programs. These files must have a .py extension. and file editing.
Example: 13. Converting a string IPython: Another enhanced Python shell with
# file my_function.py to date object tab-completion and other features.
def minmax(a,b): Eric3: A GUI Python IDE with autocompletion,
You can use the function ‘DateTime’ to convert a class browser, built-in shell and debugger.
if a <= b: string to a date object. WingIDE: Commercial Python IDE with
min, max = a, b Example: free licence available to open-source
else: from DateTime import DateTime developers everywhere.
min, max = b, a
return min, max
Module Usage
import my_function

The Python Book 67

Python essentials

Built-in 23. Do-while loops sum: This function returns the sum of all
modules elements in the list. It accepts an optional
Since Python has no do-while or do-until loop second argument: the value to start with when
19. Executing functions constructs (yet), you can use the following summing (defaults to 0).
at the time of Python method to achieve similar results:
interpreter termination while True: 28. Representing
fractional numbers
You can use ‘atexit’ module to execute functions do_something()
at the time of Python interpreter termination. if condition(): Fraction instance can be created using the
Example: following constructor:
def sum(): break Fraction([numerator
[,denominator]])
print(4+5) 24. Detecting system
def message(): platform 29. Performing
math operations
print(“Executing Now”) To execute platform-specific functions, it is very
import atexit useful to detect the platform on which the Python The ‘math’ module provides a plethora of
atexit.register(sum) interpreter is running. You can use ‘sys.platform’ mathematical functions. These work on integer
atexit.register(message) to find out the current platform. and float numbers, except complex numbers.
Output: Example: For complex numbers, a separate module is
Executing Now On Ubuntu Linux used, called ‘cmath’.
9 >>> import sys For example:
>>> sys.platform math.acos(x): Return arc cosine of
20. Converting from integer ‘linux2’ x.
to binary, hexadecimal On Mac OS X Snow Leopard math.cos(x): Returns cosine of x.
and octal >>> import sys math.factorial(x) : Returns x
>>> sys.platform factorial.
Python provides easy-to-use functions – bin(), ‘darwin’
hex() and oct() – to convert from integer to binary, 30. Working with arrays
decimal and octal format respectively. 25. Disabling and enabling
Example: garbage collection The ‘array’ module provides an efficient way to
>>> bin(24) use arrays in your programs. The ‘array’ module
‘0b11000’ Sometimes you may want to enable or disable defines the following type:
>>> hex(24) the garbage collector at runtime. You can array(typecode [, initializer])
‘0x18’ use the ‘gc’ module to enable or disable the Once you have created an array object, say
>>> oct(24) garbage collection. myarray, you can apply a bunch of methods to it.
‘030’ Example: Here are a few important ones:
>>> import gc myarray.count(x): Returns the
21. Converting any >>> gc.enable number of occurrences of x in a.
charset to UTF-8 <built-in function enable> myarray.extend(x): Appends x at the
>>> gc.disable end of the array.
You can use the following function to convert any <built-in function disable> myarray.reverse(): Reverse the
charset to UTF-8. order of the array.
data.decode(“input_charset_here”). 26. Using C-based modules
encode(‘utf-8’) for better performance 31. Sorting items

22. Removing Many Python modules ship with counterpart The ‘bisect’ module makes it very easy to keep
duplicates from lists C modules. Using these C modules will lists in any possible order. You can use the
give a significant performance boost in following functions to order lists.
If you want to remove duplicates from a list, complex applications. bisect.insort(list, item [, low [,
just put every element into a dict as a key (for Example: high]])
example with ‘none’ as value) and then check cPickle instead of Pickle, cStringIO Inserts item into list in sorted order. If item is
dict.keys(). instead of StringIO . already in the list, the new entry is inserted to
from operator import setitem the right of any existing entries.
def distinct(l): 27. Calculating maximum, bisect.insort_left(list, item [, low
minimum and sum [, high]])
d = {} out of any list or iterable Inserts item into list in sorted order. If item is
map(setitem, (d,)*len(l), l, []) already in the list, the new entry is inserted to
return d.keys() You can use the following built-in functions. the left of any existing entries.
max: Returns the largest element in the list.
min: Returns the smallest element in the list.

68 The Python Book

Python essentials

32. Using regular 34. Using SQLite database 37. Performing basic file
expression-based with Python operations (copy, delete
search and rename)
SQLite is fast becoming a very popular embedded
The ‘re’ module makes it very easy to use regxp- database because of its zero configuration You can use the module ‘shutil’ to perform basic
based searches. You can use the function needed, and superior levels of performance. You file operation at a high level. This module works
‘re.search()’ with a regexp-based expression. can use the module ‘sqlite3’ in order to work with with your regular files and so will not work with
Check out the example below. SQLite databases. special files like named pipes, block devices, and
Example: Example: so on.
>>> import re >>> import sqlite3 shutil.copy(src,dst)
>>> s = “Kunal is a bad boy” >>> connection = sqlite.connect(‘test. Copies the file src to the file or directory dst.
>>> if re.search(“K”, s): print db’) shutil.copymode(src,dst)
“Match!” # char literal >>> curs = connection.cursor() Copies the file permissions from src to dst.
... >>> curs.execute(‘’’create table item shutil.move(src,dst)
Match! ... (id integer primary key, itemno Moves a file or directory to dst.
>>> if re.search(“[@A-Z]”, s): print text unique, shutil.copytree(src, dst, symlinks
“Match!” # char class ... scancode text, descr text, price [,ignore]])
... # match either at-sign or capital real)’’’) Recursively copy an entire directory at src.
letter <sqlite3.Cursor object at 0x1004a2b30> shutil.rmtree(path [, ignore_errors
Match! [, onerror]])
>>> if re.search(“\d”, s): print 35. Working with zip files Deletes an entire directory.
“Match!” # digits class
... You can use the module ‘zipfile’ to work with 38. Executing UNIX
zip files. commands from Python
33. Working with bzip2 (.bz2) zipfile.ZipFile(filename [, mode [,
compression format compression [,allowZip64]]]) You can use module commands to execute UNIX
Open a zip file, where the file can be either a path commands. This is not available in Python 3 –
You can use the module ‘bz2’ to read and write to a file (a string) or a file-like object. instead you need to use the module ‘subprocess’.
data using the bzip2 compression algorithm. zipfile.close()¶ Example:
bz2.compress() : For bz2 Close the archive file. You must call ‘close()’ before >>> import commands
compression exiting your program or essential records will not >>> commands.getoutput(‘ls’)
bz2.decompress() : For bz2 be written. ‘bz2-example.py\ntest.py’
decompression zipfile.extract(member[, path[,
Example: pwd]]) 39. Reading environment
# File: bz2-example.py Extract a member from the archive to the current variables
import bz2 working directory; ‘member’ must be its full
MESSAGE = “Kunal is a bad boy” name (or a zipinfo object). Its file information You can use the module ‘os’ to gather operating-
compressed_message = bz2. is extracted as accurately as possible. ‘path’ system-specific information:
compress(MESSAGE) specifies a different directory to extract to. Example:
decompressed_message = bz2. ‘member’ can be a filename or a zipinfo object. >>> import os
decompress(compressed_message) ‘pwd’ is the password used for encrypted files. >>> os.path <module ‘posixpath’
print “original:”, repr(MESSAGE) from ‘/usr/lib/python2.6/posixpath.
print “compressed message:”, 36. Using UNIX-style pyc’>>>> os.environ {‘LANG’: ‘en_
repr(compressed_message) wildcards to search IN’, ‘TERM’: ‘xterm-color’, ‘SHELL’:
print “decompressed message:”, for filenames ‘/bin/bash’, ‘LESSCLOSE’:
repr(decompressed_message) ‘/usr/bin/lesspipe %s %s’,
Output: You can use the module ‘glob’ to find all the ‘XDG_SESSION_COOKIE’:
[~/src/python $:] python bz2- pathnames matching a pattern according to the ‘925c4644597c791c704656354adf56d6-
example.py rules used by the UNIX shell. *, ?, and character 1257673132.347986-1177792325’,
original: ‘Kunal is a bad boy’ ranges expressed with [ ] will be matched. ‘SHLVL’: ‘1’, ‘SSH_TTY’: ‘/dev/
compressed message: ‘BZh91AY&SY\xc4\ Example: pts/2’, ‘PWD’: ‘/home/kunal’,
x0fG\x98\x00\x00\x02\x15\x80@\x00\ >>> import glob ‘LESSOPEN’: ‘| /usr/bin
x00\x084%\x8a \x00”\x00\x0c\x84\r\ >>> glob.glob(‘./[0-9].*’) lesspipe
x03C\xa2\xb0\xd6s\xa5\xb3\x19\x00\ [‘./1.gif’, ‘./2.txt’] ......}
xf8\xbb\x92)\xc2\x84\x86 z<\xc0’ >>> glob.glob(‘*.gif’) >>> os.name
decompressed message: ‘Kunal is a [‘1.gif’, ‘card.gif’] ‘posix’
bad boy’ >>> glob.glob(‘?.gif’) >>> os.linesep
[‘1.gif’] ‘\n’

The Python Book 69

Python essentials

40. Sending email Older versions of Red Hat Linux have been moved 44. Seeding random
to the following location: ftp://archive.download. numbers
You can use the module ‘smtplib’ to send email redhat.com/pub/redhat/linux/
using an SMTP (Simple Mail Transfer Protocol) You can use the module ‘random’ to generate
client interface. 42. Launching a webpage a wide variety of random numbers. The most
smtplib.SMTP([host [, port]]) with the default web used one is ‘random.seed([x])’. It initialises
Example (send an email using Google Mail browser the basic random number generator. If x is
SMTP server): omitted or None, current system time is used;
import smtplib The ‘webbrowser’ module provides a convenient current system time is also used to initialise the
# Use your own to and from email way to launch webpages using the default generator when the module is first imported.
address web browser.
fromaddr = ‘[email protected]’ Example (launch google.co.uk with system’s 45. Working with CSV
toaddrs = ‘[email protected]’ default web browser): (comma-separated
msg = ‘I am a Python geek. Here is >>> import webbrowser values) files
the proof.!’ >>> webbrowser.open(‘http://google.
# Credentials co.uk’) CSV files are very popular for data exchange over
# Use your own Google Mail True the web. Using the module ‘csv’, you can read and
credentials while running the write CSV files.
program 43. Creating secure hashes Example:
username = ‘[email protected]’ import csv
password = ‘xxxxxxxx’ The ‘hashlib’ module supports a plethora of # write stocks data as comma-
# The actual mail send secure hash algorithms including SHA1, SHA224, separated values
server = smtplib.SMTP(‘smtp.gmail. SHA256, SHA384, SHA512 and MD5. writer = csv.writer(open(‘stocks.
com:587’) Example (create hex digest of the given text): csv’, ‘wb’, buffering=0))
# Google Mail uses secure >>> import hashlib writer.writerows([
connection for SMTP connections # sha1 Digest (‘GOOG’, ‘Google, Inc.’, 505.24, 0.47,
server.starttls() >>> hashlib.sha1(“MI6 Classified 0.09),
server.login(username,password) Information 007”).hexdigest() (‘YHOO’, ‘Yahoo! Inc.’, 27.38, 0.33,
server.sendmail(fromaddr, toaddrs, ‘e224b1543f229cc0cb935a1eb9593 1.22),
msg) 18ba1b20c85’ (‘CNET’, ‘CNET Networks, Inc.’, 8.62,
server.quit() # sha224 Digest -0.13, -1.49)
>>> hashlib.sha224(“MI6 Classified ])
41. Accessing Information 007”).hexdigest() # read stocks data, print status
FTP server ‘3d01e2f741000b0224084482f905e9b7b97 messages
7a59b480990ea8355e2c0’ stocks = csv.reader(open(‘stocks.
‘ftplib’ is a fully fledged client FTP module for # sha256 Digest csv’, ‘rb’))
Python. To establish an FTP connection, you >>> hashlib.sha256(“MI6 Classified status_labels = {-1: ‘down’, 0:
can use the following function: Information 007”).hexdigest() ‘unchanged’, 1: ‘up’}
ftplib.FTP([host [, user [, passwd ‘2fdde5733f5d47b672fcb39725991c89 for ticker, name, price, change, pct
[, acct [, timeout]]]]]) b2550707cbf4c6403e fdb33b1c19825e’ in stocks:
Example: # sha384 Digest
host = “ftp.redhat.com” >>> hashlib.sha384(“MI6 Classified status = status_
username = “anonymous” Information 007”).hexdigest() labels[cmp(float(change), 0.0)]
password = “[email protected]” ‘5c4914160f03dfbd19e14d3ec1e74bd8b99
import ftplib dc192edc138aaf7682800982488daaf540be print ‘%s is %s (%s%%)’ % (name,
import urllib2 9e0e50fc3d3a65c8b6353572d’ status, pct)
ftp_serv = ftplib. # sha512 Digest
FTP(host,username,password) >>> hashlib.sha512(“MI6 Classified 46. Installing third-
# Download the file Information 007”).hexdigest() party modules using
u = urllib2.urlopen (“ftp:// ‘a704ac3dbef6e8234578482a31d5ad29d25 setuptools
ftp.redhat.com/pub/redhat/linux/ 2c822d1f4973f49b850222edcc0a29bb89077
README”) 8aea807a0a48ee4ff8bb18566140667fbaf7 ‘setuptools’ is a Python package which lets you
# Print the file contents 3a1dc1ff192febc713d2’ download, build, install, upgrade and uninstall
print (u.read()) # MD5 Digest packages very easily.
Output: >>> hashlib.md5(“MI6 Classified
[~/src/python $:] python Information 007”).hexdigest() To use ‘setuptools’ you will need to install
ftpclient.py ‘8e2f1c52ac146f1a999a670c826f7126’ from your distribution’s package manager.
After installation you can use the command
‘easy_install’ to perform Python package
management tasks.

70 The Python Book

Python essentials

Example (installing simplejson using Third-party modules
setuptools):
kunal@ubuntu:~$ sudo easy_install 48. Generating PDF Perform the following steps to install
simplejson documents Python-Twitter:
Searching for simplejson $ wget http://python-twitter.
Reading http://pypi.python.org/simple/ ‘ReportLab’ is a very popular module for PDF googlecode.com/files/python-twitter-
simplejson/ generation from Python. 0.6.tar.gz
Reading http://undefined.org/ Perform the following steps to install ReportLab $ tar xvfz python-twitter*
python/#simplejson $ wget http://www.reportlab.org/ftp/ $ cd python-twitter*
Best match: simplejson 2.0.9 ReportLab_2_3.tar.gz $ sudo python setup.py install
Downloading http://pypi.python. $ tar xvfz ReportLab_2_3.tar.gz Example (fetching followers list):
org/packages/source/s/simplejson/ $ cd ReportLab_2_3 >>> import twitter
simplejson-2.0.9.tar.gz#md5=af5e67a39c $ sudo python setup.py install # Use you own twitter account here
a3408563411d357e6d5e47 For a successful installation, you should see a >>> mytwi = twitter.Api(username=’kunald
Processing simplejson-2.0.9.tar.gz similar message: eo’,password=’xxxxxx’)
Running simplejson-2.0.9/setup.py ############SUMMARY INFO########### >>> friends = mytwi.GetFriends()
-q bdist_egg --dist-dir /tmp/easy_ ################################### >>> print [u.name for u in friends]
install-FiyfNL/simplejson-2.0.9/egg- #Attempting install of _rl_accel, sgmlop [u’Matt Legend Gemmell’, u’jono wells’,
dist-tmp-3YwsGV & pyHnj u’The MDN Big Blog’, u’Manish Mandal’,
Adding simplejson 2.0.9 to easy- #extensions from ‘/home/kunal/python/ u’iH8sn0w’, u’IndianVideoGamer.com’,
install.pth file ReportLab_2_3/src/rl_addons/rl_accel’ u’FakeAaron Hillegass’, u’ChaosCode’,
Installed /usr/local/lib/python2.6/ ################################### u’nileshp’, u’Frank Jennings’,..’]
dist-packages/simplejson-2.0.9-py2.6- #Attempting install of _renderPM
linux-i686.egg #extensions from ‘/home/kunal/python/ 50. Doing Yahoo! news
Processing dependencies for simplejson ReportLab_2_3/src/rl_addons/renderPM’ search
Finished processing dependencies for # installing with freetype version 21
simplejson ################################### You can use the Yahoo! search SDK to access
Example: Yahoo! search APIs from Python.
47. Logging to system log >>> from reportlab.pdfgen.canvas import Perform the following steps to install it:
Canvas $wget http://developer.yahoo.com/
You can use the module ‘syslog’ to write to system # Select the canvas of letter page size download/files/yws-2.12.zip
log. ‘syslog’ acts as an interface to UNIX syslog >>> from reportlab.lib.pagesizes import $ unzip yws*
library routines. letter $ cd yws*/Python/pYsearch*/
Example: >>> pdf = Canvas(“bond.pdf”, pagesize = $ sudo python setup.py install
import syslog letter) Example:
syslog.syslog(‘mygeekapp: started # import units # Importing news search API
logging’) >>> from reportlab.lib.units import cm, >>> from yahoo.search.news import
for a in [‘a’, ‘b’, ‘c’]: mm, inch, pica NewsSearch
>>> pdf.setFont(“Courier”, 60) >>> srch = NewsSearch(‘YahooDemo’,
b = ‘mygeekapp: I found letter ‘+a >>> pdf.setFillColorRGB(1, 0, 0) query=’London’)
syslog.syslog(b) >>> pdf.drawCentredString(letter[0] / 2, # Fetch Results
syslog.syslog(‘mygeekapp: the script inch * 6, “MI6 CLASSIFIED”) >>> info = srch.parse_results()
goes to sleep now, bye,bye!’) >>> pdf.setFont(“Courier”, 40) >>> info.total_results_available
Output: >>> pdf.drawCentredString(letter[0] / 2, 41640
$ python mylog.py inch * 5, “For 007’s Eyes Only”) >>> info.total_results_returned
$ tail -f /var/log/messages # Close the drawing for current page 10
Nov 8 17:14:34 ubuntu -- MARK -- >>> pdf.showPage() >>> for result in info.results:
Nov 8 17:22:34 ubuntu python: # Save the pdf page ... print “’%s’, from %s” %
mygeekapp: started logging >>> pdf.save() (result[‘Title’], result[‘NewsSource’])
Nov 8 17:22:34 ubuntu python: Output: ...
mygeekapp: I found letter a @image:pdf.png ‘Afghan Handover to Be Planned at London
Nov 8 17:22:34 ubuntu python: @title: PDF Output Conference, Brown Says’, from Bloomberg
mygeekapp: I found letter b .................
Nov 8 17:22:34 ubuntu python: 49. Using Twitter API
mygeekapp: I found letter c
Nov 8 17:22:34 ubuntu python: You can connect to Twitter using the ‘Python-
mygeekapp: the script goes to sleep Twitter’ module.
now, bye,bye!

The Python Book 71

Work

Pywiththon

74 Python for professionals

Put your skills to professional use

82 Extensions for XBMC

Enhance XBMC with this tutorial

88 Scienti c computing

Get to grips with NumPy

92 Instant messaging

Get chatting using Python

98 Replace your shell

Use Python for your primary shell

102 Python for system admins

How Python helps system administration

92

“With Python, you can tweak
and realise your ideal system set-up”

72 The Python Book

74 82 88

The Python Book 73

Work with Python

PYTHON FOR

PROFESSIONALS

Python is relied upon by web developers, engineers and
academic researchers across the world. Here’s how to put your
Python skills to professional use

74 The Python Book

Work with Python

System administration

Get the most out of Python in handling all of the day-to-day
upkeep that keeps your system healthy

System administration tasks are some of the most SYSTEM ADMINISTRATION Left Python scripts
annoying things that you need to deal with when you BASH, PERL, PYTHON enable you to instruct
have to maintain your own system. Because of this, and interact with your
system administrators have constantly been trying to find OPERATING SYSTEM operating system
ways to automate these types of tasks to maximise their
time. They started with basic shell scripts, and then moved CPU FILES/IO
on to various scripting languages. For a long time, Perl had
been the language of choice for developing these types way to do this is to edit the values directly within the
of maintenance tools. However, Python is now growing in environs mapping.
popularity as the language to use. It has reached the point
where most Linux distributions have a Python interpreter Another category of tasks you may want to automate
included in order to run system scripts, so you shouldn’t is when working with files. For example, you can get the
have any excuse for not writing your own scripts. current working directory with code like

Because you will be doing a lot system level work, you
will have most need of a couple of key Python modules.
The first module is ‘os’. This module provides the bulk of
the interfaces to interacting with the underlying system.
The usual first step is to look at the environment your
script is running in to see what information might exist
there to help guide your script. The following code gives
you a mapping object where you can interact with the
environment variables active right now:

import os cwd = os.getcwd()
os.environ
You can then get a list of the files in this directory with
You can get a list of the available environment variables
with the function “os.environs.keys()”, and then access os.listdir(cwd)
individual variables with “os.environs[key]”. These
environment variables are used when you spawn a You can move around the file system with the function
subprocess, as well. So you will want to change values, “os.chdir(new_path)”. Once you’ve found the file you are
like the PATH or the current working directory, in order interested in, you can open it with “os.open()” and open it
for you to run these subprocesses correctly. While there for reading, writing and/or appending. You can then read
is a “putenv” function that edits these values for you, it or write to it with the functions “os.read()” and “os.write()”.
unfortunately does not exist on all systems. So the better Once done, you can close the file with “os.close()”.

Running subprocesses from Python Scheduling
with cron
The underlying philosophy of Unix is to build small, specialised This provides you with a long file listing for the current
programs that do one job extremely well. You then chain these directory. The function “run()” was introduced in Python Once you have your script
together to build more complex behaviours. There is no reason 3.5 and is the suggested way of handling this. If you have an all written up, you may want
why you shouldn’t use the same philosophy within your Python older version, or if you require more control than that, then to schedule them to run
scripts. There are several utility programs available to use with you can employ the underlying “popen()” function that we automatically without your
very little work on your part. The older way of handling this was mentioned earlier instead. If you want to get the output, you intervention. On Unix systems,
through using functions like “popen()” and “spawnl()” from the can use the following: you can have cron run your
os module, but a better way of running other programs is by script on whatever schedule
using the subprocess module instead. You can then launch a cmd_output = subprocess.run([‘ls’, ‘-l’], is necessary. The utility
program, like ls, by using: stdout=subprocess.PIPE) “crontab -l” lists the current
contents of your cron file, and
import subprocess The variable “cmd_output” is a CompletedProcess object that “crontab -e” lets you edit the
subprocess.run([‘ls’, ‘-l’]) contains the return code and a string holding the stdout output. scheduled jobs that you want
cron to run.

The Python Book 75

Work with Python

Web development

Python has several frameworks available for all of your
various web development tasks. We will look at some of the
more popular ones

With the content and the bulk of the computing hosted USER Left Python interpreters work
on a server, a web application can better guarantee with your databases to power a
a consistent experience for the end user. The popular WEB SERVER web server
Django framework provides a very complete environment Bottom The Model-View-
of plugins and works on the DRY principle (Don’t Repeat DATABASE PYTHON Controller architecture is often
Yourself). Because of this, you should be able to build INTERPRETER used for UIs
your web application very quickly. Since Django is built
on Python, you should be able to install it with “sudo pip python manage.py runserver Virtual
install Django”. Most distributions should have a package environments
for Django, too. Depending on what you want to do with This will start up a server listening to port 8000 on your
your app, you may need to install a database like MySQL or local machine. Because this built in server is designed to When you start developing
postgresql to store your application data. be used for development, it automatically reloads your your own applications, you
Python code for each request. This means that you don’t may begin a descent into
There are Django utilities available to automatically need to restart the server to see your code changes. dependency hell. Several
generate a starting point for your new project’s code: Python packages depend
All of these steps get you to a working project. You are on other Python packages.
django-admin startproject newsite now ready to start developing your applications. Within the This is its strength, but also
“newsite” subdirectory, you can type: its weakness. Luckily, you
This command creates a file named “manage.py” and have virtualenv available
a subdirectory named “newsite”. The file “manage.py” python manage.py startapp newapp to help tame this jungle.
contains several utility functions you can use to administer You can create new virtual
your new application. The newly created subdirectory This will create a new subdirectory named “newapp”, with environments for each of your
contains the files “__init__.py”, “settings.py”, “urls.py” and the files “models.py”, “tests.py” and “views.py”, among projects. Thankfully with this,
“wsgi.py”. These files and the subdirectory they reside in others. The simplest possible view consists of the code: you can be sure to capture all
comprise a Python package that gets loaded when your of the dependencies for your
web site is started up. The core configuration for your site from django.http import HttpResponse own package.
can be found in the file “settings.py”. The URL declarations, def index(request):
basically a table of contents for your site, are stored in the
file “urls.py”. The file “wsgi.py” contains an entry point for return HttpResponse(“Hello world”)
WSGI-compatible web servers.
This isn’t enough to make it available, however. You will
Once your application is done, it should be hosted on a also need to create a URLconf for the view. If the file
properly configured and hardened web server. But, this is “urls.py” doesn’t exist yet, create it and then add the code:
inconvenient if you are in the process of developing your
web application. To help you out, Django has a web server from django.conf.urls import url
built into the framework. You can start it up by changing from . Import views
directory to the “newsite” project directory and running the
following command: urlpatterns = [ url(r’^$’, views.index,
name=‘index’), ]
MODEL
The last step is to get the URL registered within your
Manipulates Updates project. You can do this with the code

VIEW CONTROLLER from django.conf.urls import include, url
from django.contrib import admin
Sees Uses urlpatterns = [ url(r’^newapp/’, include(‘newapp.
urls’)),
USER

76 The Python Book

Using the PyCharm IDE Work with Python

THE EDITOR PANE

The main editor pane can be configured to match your
own style, or the style of one of the other main editors,
like emacs. It handles syntax highlighting, and even
displays error locations

THE PROJECT PANE

This pane is the central location for your project. All of
your files and libraries are located here. Right-clicking
in the pane brings up a drop-down menu where you can
add new files or libraries, run unit tests, or even start up
a debugger

THE STATUS BARE

PyCharm does a lot of work behind the scenes.
The status bar helps you keep track of all of these
background processes

url(r’^admin’, admin.site.urls), ] Other Python
frameworks
This needs to be put in the “urls.py” file for the main
project. You can now pull up your newly created While Django is one of the most popular frameworks around
application with the URL http://localhost:8000/newapp/. for doing web development, it is by no means the only one
around. There are several others available that may prove to
The last part of many applications is the database be a better fit for particular problem domains. For example,
side. The actual connection details to the database, like if you are looking for a really self-contained framework, you
the username and password, are contained in the file could look at web2py. Everything you need to be able to have
“settings.py”. This connection information is used for a complete system, from databases to web servers to a
all of the applications that exist within the same project. ticketing system, are included as part of the framework. It is
You can create the core database tables for your site with so self-contained that it can even run from a USB drive
this command:
If you need even less of a framework, there are several
python manage.py migrate mini-frameworks that are available. For example, CherryPy
is a purely Pythonic multi-threaded web server that you
Terminal For your own applications, you can define the data model can embed within your own application. This is actually
development you need within the file “models.py”. Once the data the server included with TurboGears and web2py. A really
environments model is created, you can add your application to the popular microframework is a project called flask. It includes
INSTALLED_APPS section of the “settings.py” so that integrated unit testing support, jinja2 templating and RESTful
django knows to include it in any database activity. You request dispatching.
initialize it with:
One of the oldest frameworks around is zope, now up to
python manage.py makemigrations newapp version 3. This latest version was renamed BlueBream. Zope
is fairly low-level, however. You may be more interested in
When you are in the middle of Once these migrations have been created, you need to looking at some of the other frameworks that are built on
developing your application, apply them to the database by using the command: top of what is provided by zope. For example, pyramid is a
you may need to have several very fast, easy to use framework that focuses on the most
different terminal windows python manage.py migrate essential functions required by most web applications. To
open in order to have a code this end, it provides templating, the serving of static content,
editor open, a monitor on the Any time you make changes to your model, you will need to mapping of URLs to code, among other functions. It handles
server, and potentially testing run the makemigrations and migrate steps again. this while providing tools for application security.
output. If you are doing this on
your own machine, this isn’t an Once you have your application finished, you can make If you are looking for some ideas, there are several open
issue. But, if you are working the move to the final hosting server. Don’t forget to check source projects that have been built using these frameworks,
remotely, you should look into the available code within the Django framework before from blogs, to forums to ticketing systems. These projects can
using tmux. It can provide a putting too much work into developing your own code. provide some best-practices when you go to construct your
much more robust terminal own application.
environment for you.

The Python Book 77

Work with Python

Computational science

Python is fast becoming the go-to language for
computational science

Python has become one of the key languages used in So, not only is it faster, it is also written in a shorter, clearer Left The numpy package
science. There is a huge number of packages available form. Along with the new datatype, numpy provides makes it simple to visualise
to handle almost any task that you may have and, overloaded forms of all of the operators that are of your data
importantly, Python knows what it isn’t good at. To deal most use, like multiplication or division. It also provides
with this, Python has been designed to easily incorporate optimised versions of several functions, like the trig Parallel
code from C or FORTRAN. This way, you can offload any functions, to take advantage of this new datatype. Python
heavy computations to more efficient code.
The largest package available, that is built on top of One of the really powerful
The core package of most of the scientific code numpy, is scipy. Scipy provides sub-sections in several parts of Ipython (or jupyter)
available is numpy. One of the problems in Python is that areas of science. Each of these sub-sections need is that it is built with a client/
the object oriented nature of the language is the source to be imported individually after importing the main server model. This means that
of its inefficiencies. With no strict types, Python always scipy package. For example, if you are doing work with it is relatively easy to setup
needs to check parameters on every operation. Numpy multiple machines to act as
provides a new datatype, the array, which helps solve a server pool. You can then
some of these issues. Arrays can only hold one type of farm out multiple tasks to
object, and because Python knows this it can use some these other machines to get
optimisations to speed things up to almost what you can even more work done. While
get from writing your code directly in C or FORTRAN. The this doesn’t run any particular
classic example of the difference is the for loop. Lets say function in parallel, it does let
you wanted to scale a vector by some value, something like you run longer functions in the
a*b. In regular Python, this would look like background while you work on
something else.
for elem in b:
c.append(a * elem)

In numpy, this would look like:

a*b

Spyder, the IDE for scientists

THE EDITOR PANE VARIABLE EXPLORER

This pane is where you can open and edit your source The variable explorer pane lets you access all of the
files. Above this pane are buttons to allow you to simply data structures within the current Python interpreter.
run the code, or run it under a debugger. Under the You need to actually run your code for anything to show
debugger, you can set breakpoints and step through up here
each line of code individually

IPYTHON CONSOLE

The console window lets you interact directly with the
underlying interpreter that will be used when you try and
run your code

78 The Python Book

Work with Python

Interactive science
with jupyter

Above The ability to generate For a lot of scientific problems, you need to play with your
complex plots is essential data in an interactive way. The original way you would do
this was to use the Ipython web notebook. This project has
differential equations, you can use the “integrate” section since been renamed Jupyter. For those who have used a
to solve them with code that looks like program like Mathematica or Maple, the interface should
seem very familiar. Jupyter starts a server process, by
default on port 8888, and then will open a web browser
where you can open a worksheet. Like most other programs
of this type, the entries run in chronological order, not in
the order that they happen on the worksheet. This can
be a bit confusing at first, but it means that if you go to
edit an earlier entry, all of the following entries need to be
re-executed manually in order to propagate that change
through the rest of the computations.

Jupyter has support for pretty printing math within
the produced web page. You can also mix documentation
blocks and code blocks within the same page. This means
that you can use it to produce very powerful educational
material, where students can read about some technique,
and then actually run it and see it in action. By default,
Jupyter will also embed matplotlib plots within the same
worksheet as a results section, so you can see a graph of
some data along with the code that generated it. This is
huge in the growing need for reproducible science. You can
always go back and see how any analysis was done and be
able to reproduce any result at all.

import scipy
import scipy.integrate
result = scipy.integrate.quad(lambda x: sin(x), 0,
4.5)

The need for Differential equations crop up in almost every scientific Above Jupyter Notebook is a web application that is used
speed field. You can do statistical analysis with the “stats” for creating and sharing documents that contain live code
section. If you want to do some signal processing, you can and equations
Sometimes you need as much use the “signal” section and the “fftpack” section. This
speed as your are capable of package is definitely the first stop for anyone wanting to do what variables are valid to be considered in your equations.
pushing on your hardware. In any scientific processing. You can then start doing manipulations using these
these cases, you always have registered variables.
the option of using Cython. Once you have collected your data, you usually need
This lets you take C code from to graph it, in order to get a visual impression of patterns You may have large amounts of data that you need
some other project, which within it. The primary package you can use for this is to work with and analyze. If so, you can use the pandas
has probably already been matplotlib. If you have ever used the graphics package package to help deal with that. Pandas has support
optimised, and use it within in R before, the core design of matplotlib has borrowed for several different file formats, like CSV files, Excel
your own Python program. In quite a few ideas. There are two categories of functions for spreadsheets or HDF5. You can merge and join datasets,
scientific programming, you graphing, low-level and high-level. High-level functions try or do slicing or subsetting. In order to get the best
are likely to have access to to take care of as many of the menial tasks, like creating a performance out of the code, the heaviest lifting is done by
code that has been worked plot window, drawing axes, selecting a coordinate system, Cython code that incorporates functions written in C. Quite
on for decades and is highly as possible. The low-level functions give you control over a few ideas on how to manipulate your data was borrowed
specialised. There is no need almost every part of a plot, from drawing individual pixels from how things are done in R.
to redo the development to controlling every aspect of the plot window. It also
effort that has gone into it. borrowed the idea of drawing graphs into a memory based You now have no reason not to start using Python for
window. This means that it can draw graphs while running your scientific work. You should be able to use it for almost
on a cluster. any problem that comes up!

If you need to do symbolic math, you may be more used
to using something like Mathematica or Maple. Luckily,
you have sympy that can be used to do many of the same
things. You can use Python to do symbolic calculus, or to
solve algebraic equations. The one weird part of sympy is
that you need to use the “symbols()” function to tell sympy

The Python Book 79

Work with Python

Robotics and electronics

Robotics is the most direct interface between your code and
the real world around you

Robotics is the most direct way that your code can
interact with the world around you. It can read actual
sensor information and move real actuators and get real
work done.

The first thing your robot needs is the ability to sense
the world around it. The one sense that we as humans feel
is most useful is sight. With web cameras being so cheap
and easy to connect to hardware, vision is easy to give to
your robot. The real problem is how to interpret this data.
Luckily, you can use the OpenCV project to do just that. It is
a vision package that can provide simple image gathering
and processing, to extremely complex functions like face
recognition and extraction of 3D objects. You can identify
and track objects moving through your field of view. You
can also use OpenCV to give you robot some reasoning
capabilities, too. OpenCV includes a set of functions

for machine learning, where you can do statistical Arduino
classification or data clustering, and use it to feed decision
trees or even neural networks. In contrast to the Raspberry
Pi, which runs a full OS from
Another important sense that you may want to use is its SD card, the Arduino
sound. The jasper project is one that is developing a boards are microcontrollers
rather than complete
complete voice control system. This project would computers. Instead of
give you the structure you need to give your robot running an OS, the Arduino
the ability to listen for and respond to your verbal platform executes code that
commands. The project has gotten to the point where is interpreted by its firmware.
you can give it a command and the voice recognition It is mainly used to interface
software can translate this into text. You then need to with hardware such as motors
build a mapping of what pieces of text correspond to what and servos, sensors, and
commands to execute. devices such as LEDs, and
There are lots of other sensors you could have, but this is incredibly capable in this
begins to leave the realm of store-bought hardware. Most regard. Arduinos are widely
other sensors, like temperature, pressure, orientation used in robotics projects
or location, need specialised hardware that needs to and can be a powerful
be interfaced to the computer brain for your robot. This complement to the Pi.

Raspberry Pi ROS – Robot Operating System

While we haven’t discussed While you could simply write some code that runs on a Because ROS is a complete operating system, rather than
what kind of computer to use standard computer and a standard Linux distribution, this a library, it is wrong to think that you can use it in your Python
for your robotics project, you is usually not optimal when trying to handle all of the data code. It is better to think that you can write Python code
should consider the famous processing that a robot needs when dealing with events in that can be used in ROS. The fundamental design is to be as
Raspberry Pi. This tiny realtime. When you reach this point, you may need to look at agnostic as possible. This means that interfaces to your code
computer should be small a dedicated operating system – the Robot Operating System should be clean and not particularly care where they running
enough to fit into almost (ROS). ROS is designed to provide the same type of interface or who is talking to them. Then, it can be used within the graph
any robot structure that you between running code the computer hardware it is running of processes running within ROS. There are standard libraries
might be building. Since it is on, with the lowest possible overhead. One of the really available that allow you to do coordinate transformations,
already running Linux and powerful features of ROS is that it is designed to facilitate useful for figuring out where sensors or limbs are in space.
Python, you should be able communication between different processes running on the There is a library available for creating preemptible tasks for
to simply copy your code computer, or potentially over multiple computers connected data processing, and another for creating and managing the
development work to the Pi. over some type of network. Instead of each process being a types of messages that can be handed around the various
It also includes its own IO bus silo that is protected from all other processes, ROS is more of processes. For extremely time-sensitive tasks, there is a
so that you can have it read a graph of processes with messages being passed between plugin library that allows you to write a C++ plugin that can be
it’s own sensors. them all. loaded within ROS packages.

80 The Python Book

Work with Python

For low-level work, check out Arduinos

THE MAIN EDITOR

You have access to a large number of libraries,
and support for a large number of versions of the
Arduino boards. The code is essentially C, so Python
programmers shouldn’t be too far out of their depths

OUTPUT WINDOW

This pane contains output from various tasks. This
might be compiling the source code, or uploading it to
the Arduino board being used in your project

THE STATUS BAR

The status bar reminds you which type of board your
are currently programming for, as well as which port the
Arduino IDE thinks it is on. Always verify this information
before trying to upload your control program to the
board in question

means it is time to get your soldering iron out. As for Luckily, the Arduino is designed to connect to the serial Bypassing
reading the data in, this is most often done over a basic port of your computer, so you can simply use pySerial to the GIL
serial connection. You can then use the pySerial module to talk to it. You can send commands to code that you have
connect to the serial port and read data off the connection. written and uploaded to the Arduino to handle the actual For robotics work, you may
You can use: manipulations of the various actuators. The Arduino need to run some code truly
can talk back, however. This means that you can read in parallel, on multiple CPUs.
import serial feedback data to see what effect your movements have Python currently has the GIL,
had. Did you end up turning your wheels as far as you which means that there is a
to load the module and start communicating with your wanted to? This means that you could also use the Arduino fundamental bottleneck built
sensor. The problem is that this is a very low-level way to as an interface between your sensors and the computer, into the interpreter. One way
communicate. You, as the programmer, are responsible for thus simplifying your Python code even more. There are around this is to actually run
all of the details. This includes communication speed, byte loads of add-on modules available, too, that might be able multiple Python interpreters,
size, flow control; basically everything. So this will definitely to provide the sensing capabilities that you require straight one for each thread of
be an area of your code where you should plan on spending out of the box. There are also several models of Arduino, so execution. The other option
some debugging time. you may be able to find a specialised model that best fits is to move from Cpython to
your needs. either Jython or IronPython, as
Now that you have all of this data coming in, what will neither has a GIL.
you do with it? You need to be able to move actuators out Now that you have all of this data coming in and the
in the world and have real effects. This could be motors ability to act out in the real world, the last step is giving
for wheels or tracks, levers to shift objects, or potentially your robot some brains. This is where the state of the art
complete limbs, like arms or legs. While you could try and unfortunately does not live up to the fantasy of R2-D2 or
drive these types of electronic devices directly from the C-3P0. Most of your actual innovative coding work will
output ports of your computer, there usually isn’t enough likely take place in this section of the robot. The general
current available to provide the necessary power. So, term for this is artificial intelligence. There are several
you will need to have some off-board brains capable of projects currently underway that you could use as a
handling the supplying of power to these devices. One of starting point to giving your robot some real reasoning
the most popular candidates for this task is the Arduino. capability, like SimpleAI or PyBrain.

The Python Book 81

Work with Python

Current media Rating (only available for
selection hosted plug-ins)

List of
installed
plug-ins

Configure Localised
launcher description string

Make extensions for Opens changelog
XBMC with Python for the plug-in

Python is the world’s most popular easy-to-use open source
language. Learn how to use it to build your own features for
XBMC, the world’s favourite FOSS media centre

Resources all major platforms, including different hardware the look and feel of just about every facet of
architectures. It is available for Linux, Windows, the package.
XBMC: www.xbmc.org/download Mac OS X, Android, iOS and Raspberry Pi.
Python 2.7x Depending upon which category your
Python IDE (optional) In these pages we will learn to build extensions extension fits, you will have to create the
Code on FileSilo for XBMC. Extensions are a way of adding extension directory accordingly. For example…
features to XBMC without having to learn the
XBMC is perhaps the most important thing that core of XBMC or alter that core in any way. One Plug-ins:
has ever happened in the open source media additional advantage is that XBMC uses Python plugin.audio.ludaudi: An audio plug-in
centre space. It started its life on the original as its scripting language, and this can be also plugin.video.ludvidi: A video plug-in
Xbox videogames console and since then it has used to build the extensions. This really helps script.xxx.xxx: A program
become the de facto software for multimedia new developers get involved in the project since
aficionados. It also has been forked into many Python is easy to learn compared to languages In this tutorial we will build an XBMC plug-in
other successful media centre applications such like C/C++ (from which the core of XBMC is made). called LUD Entertainer. This plug-in will provide a
as Boxee and Plex. XBMC has ultimately grown nice way to watch videos from Reddit from within
into a very powerful open source application with XBMC supports various types of extensions (or XBMC. Our plug-in will show various content such
a solid community behind it. It supports almost Add-ons): Plugins, Programs and Skins. Plugins as trailers and documentaries from Reddit. We’ll
add features to XBMC. Depending on the type also allow our users to add their own Subreddit.
of feature, a plug-in will appear in the relevant Each video can then be categorised as Hot, New,
media section of XBMC. For example, a YouTube Top, Controversial etc. With this plug-in we will
plug-in would appear in the Videos section. demonstrate how easy it is hook into XBMC’s
Scripts/Programs are like mini-applications for built-in method to achieve a very high-quality
XBMC. They appear in the Programs section. user experience.
Skins are important since XBMC is a completely
customisable application – you can change Due to space limitations, we aren’t able to print
the full code here. We recommend downloading
the complete code from FileSilo.

82 The Python Book

Work with Python

01 Preparing the directory structure version="2.1.0"/> 05 Setting up plug-in metadata
As we have mentioned previously, each <import addon="plugin.video. Metadata about the plug-in is provided in
XBMC extension type follows a certain directory youtube" version="3.0.0"/> <extension point="xbmc.addon.metadata">. The
naming convention. In this case we are building <import addon="plugin.video.vimeo" following are the important elements…
a video plug-in, so the plug-in directory name version="2.3.0"/>
would be plugin.video.ludlent. But that’s just the <import addon="plugin.video. <platform>: Most of the time, XBMC extensions
root directory name – we will need several other dailymotion_com" version="1.0.0"/> are cross-platform compatible. However, if you
folders and files as well. </requires> depend on the native platform library that is only
The following describes the directory structure of available on certain platforms then you will need
LUD Linux Entertainer: In the above code we have added a dependency to set the supported platforms here. Accepted
plugin.video.ludent – Root Plugin directory to a library called xbmc.python version values for the platform are: all, linux, osx, osx32,
|-- addon.xml 2.1. Currently it is added as a mandatory osx64, ios (Apple iOS) , windx (Windows DirectX),
|-- changelog.txt dependency. To make the dependency wingl (Windows OpenGL) and android.
|-- default.py optional you will need to add optional="true";
|-- icon.png eg <import addon="kunal.special" <summary lang="en">: This gives a brief
|-- LICENSE.txt version="0.1.0" optional="true" /> description of the plug-in. Our example sets the
|-- README language attribute as English, but you can use
`-- resources In the above example we have added core other languages too.
dependency xbmc.python to 2.1.0 because it’s <description>: A detailed description of the
|-- lib the version shipped with XBMC version Frodo plug-in.
`-- settings.xml 12.0 and 12.1 . If you were to add xbmc.python <website>: Webpage where the plug-in is hosted.
to 2.0 then it would only work in XBMC Eden 11.0 <source>: Source code repository URL. If you are
02 Creating addon.xml and not in the latest version. hosting your plug-in on GitHub, you can mention
An addon.xml file needs to be created in the repository URL here.
the root of the extension directory. The addon.xml For the current version of XBMC 12.1, the <forum>: Discussion forum URL for your plug-in.
file contains the primary metadata from a XBMC following versions of core XBMC components <email>: Author email. You can directly type email
extension. It contains overview, credits, version are shipped: or use a bot-friendly email address like max at
information and dependencies information about xbmc.python 2.1.0 domain dot com.
the extension. xbmc.gui 4.0.0
xbmc.json 6.0.0 06 Setting changelog, icon, fanart
The root element of addon.xml is the <addon> xbmc.metadata 2.1.0 and licence
element. It is defined as: xbmc.addon 12.0.0 We need a few additional files in the plug-in
directory…
<addon id="plugin.video. In addition to xbmc.python we are also adding
ludent" name="LUD HSW Viewer" some third-party plug-ins as dependencies, changelog.txt: You should list the changes made
version="0.0.1" provider- such as plugin.video.youtube. These plug-ins to your plug-in between releases. The changelog
name="LUDK"> will be installed automatically when we install is visible from the XBMC UI.
rest of the content is placed here plugin.video.ludent.
</addon> An example changelog:
04 Setting up the provider and 0.0.1
Here, id is the identifier for the plug-in, so entry point - Initial Release
it should be unique among all the XBMC Our extension is supposed to provide the video 0.0.2
extensions, and id is also used for the directory content for XBMC. In order to convey that, we - Fixed Video Buffering Issue
name; version tells XBMC the extension have to set up the following element:
version number, which helps in its ability to icon.png: This will represent the plug-in in the
deliver automatic updates – XBMC follows the <extension point="xbmc.python. XBMC UI. It needs to be a non-transparent PNG
Major.Minor.Patch versioning convention; name is pluginsource" library="default. file of size 256x256.
the English title of the plug-in. py">
<provides>video</provides> fanart.jpg (optional): The fanart.jpg is rendered
Note: Steps 3 to 5 cover entries that need to be </extension> in the background if a user selects the plug-in
added within the addon.xml file. in XBMC. The art needs to be rendered in HDTV
Here, the library attribute sets up the plug-in formats, so its size can range from 1280x720
03 Adding dependency information entry point. In this example default.py will be (720p) up to the maximum 1920x1080 (1080p).
Dependency inside an extension is executed when the user activates the plug-in.
managed using the <requires> element. The <provides> elements sets up the media
type it provides. This also gets reflected in the
<requires> placement of the plug-in. Since ours is a video
<import addon="xbmc.python" plug-in, it will show up in the Videos section
of XBMC.

The Python Book 83

Work with Python

License.txt: This file contains the licence of The following are a few important settings resource/language/english/string.xml
the distributed plug-in. The XBMC project types that you can use… example:
recommends the use of the Creative Commons text: Used for basic string inputs.
Attribution-ShareAlike 3.0 licence for skins, ipaddress: Used to collect internet addresses. <?xml version="1.0" encoding="utf-8"
and GPL 2.0 for add-ons. However, most of the number: Allows you enter a number. XBMC will standalone="yes"?>
copyleft licences can be used. also provide an on-screen numeric keyboard for <strings>
the input. <string id="30001">Add subreddit</
Note: For the purpose of packaging, extensions/ slider: This provides an elegant way to collect string>
add-ons/themes/plug-ins are the same. integer, float and percentage values. You can get <string id="30002">Hot</string>
the slider setting in the following format: <string id="30003">New</string>
07 Providing settings for the plug-in <string id="30004">Top</string>
Settings can be provided by the file <setting label="21223" type="slider" <string id="30005">Controversial</
resources/settings.xml. These are great for user- id="sideinput" default="10" string>
configurable options. range="1,1,10" option="int" /> <string id="30006">Hour</string>
<string id="30007">Day</string>
Partial: resources/settings.xml In the above example we are creating a slider with <string id="30008">Week</string>
min range 1, max range 10 and step as 1. In the <string id="30009">Month</string>
<settings> option field we are stating the data type we are <string id="30010">Year</string>
<category label="30109"> interested in – we can also set option to "float" </strings>
<setting id="filter" type="bool" or "percent".
label="30101" default="false"/> As you may have seen in the settings.xml
<setting type="sep" /> bool: Provides bool selection in the form of on example, all the labels are referring to string
<setting id="showAll" type="bool" or off. ids. You can have many other languages as
label="30106" default="false"/> file: Provides a way to input file paths. XBMC will well. Depending upon the language XBMC is
<setting id="showUnwatched" provide a file browser to make the selection of file. running in, the correct language file will be
type="bool" label="30107" If you are looking to make selection for a specific loaded automatically.
default="true"/> type of file you can use audio, video, image or
<setting id="showUnfinished" executable instead of file. Post XBMC Frodo (12.1), strings.xml will be
type="bool" label="30108" deprecated. Post Frodo, XBMC will be moved
default="false"/> folder: Provides a way to browse for a folder… to a GNU gettext-based translation system;
<setting type="sep" /> gettext uses PO files. You can use a tool called
<setting id="forceViewMode" Example: xbmc-xml2po to convert strings.xml into
type="bool" label="30102" <setting label="12001" type="folder" equivalent PO files.
default="true"/> id="folder" source="auto"
<setting id="viewMode" type="number" option="writeable"/> 09 Building default.py
label="30103" default="504"/> Here, source sets the start location for the Since our plug-in is small, it will all be
</category> folder, while option sets the write parameter for contained inside default.py. If you are developing
<category label="30110"> the application. a more complex add-on then you can create
<setting id="cat_hot" type="bool" supporting files in the same directory. If your
label="30002" default="true"/> sep & lsep: sep is used to draw a horizontal line library depends upon third-party libraries, you
<setting id="cat_new" type="bool" in the setting dialog; lsep is used for drawing have two ways to go about it. You can either place
label="30003" default="true"/> a horizontal line with text. They do not collect the third-party libraries into the resources/lib
</category> any input but are there for building better user folder; or bundle the library itself into a plug-in,
</settings> interface elements… then add that plug-in as the dependency in the
addon.xml file.
Here, label defines the language id string which <setting label="21212" type="lsep"
will then be used to display the label. id defines /> Our plug-in works with reddit.tv. This is the
the name which will be used for programmatic website from Reddit which contains trending
access. type defines the data type you want 08 Language support videos shared by its readers. Videos posted on
to collect; it also affects the UI which will be Language support is provided in Reddit are actually sourced from YouTube, Vimeo
displayed for the element. default defines the the form of the strings.xml file located in and Dailymotion.
default value for the setting. You should always resources/languages/[language name]. This
use a default value wherever possible to provide a approach is very similar to many large software We will be starting off default.py using the
better user experience. projects, including Android, where static strings following imports:
are never used.
import urllib
import urllib2
…
import xbmcplugin

84 The Python Book

Work with Python

import xbmcgui idFile idPath strFilename playCount lastPlayed dateAdded
import xbmcaddon 1 1 2013-08-06 23:47
2 2 plugin://plugin. 2013-08-07 22:42
Apart from xbmcplugin, xbmcgui and 3 2 2013-08-08 00:09
xbmcaddon, the rest are all standard Python 4 2 plugin://plugin. 1 2013-08-08 00:55
libraries which are available on PyPI (Python 5 2 2013-08-08 00:58
Package Index) via pip. You will not need to install plugin://plugin. 1
any library yourself since the Python runtime for
XBMC has all the components built in. plugin://plugin. 1

urllib and urllib2 help in HTTP communication. plugin://plugin. 1
socket is used for network I/O; re is used
for regular expression matching; sqlite3 is 12 Building helper functions The above table is an example of a files table.
the Python module for accessing an SQLite In this step we will look at some of the
embedded database; xbmcplugin, xbmcgui and important helper functions. addSubreddit(): Our plug-in allows users to add
xbmcaddon contain the XBMC-specific routine. their own Subreddit. This function takes the
getDbPath(): This returns the location of the Subreddit input from the user, then saves it in
10 Initialising SQLite database file for videos. XBMC stores the subreddits file inside the addon data folder.
During the initialisation process, we will library and playback information in SQLite DB
be reading various settings from settings.xml. files. There are separate databases for videos The following sets the subreddits file location:
Settings can be read in the following way: and music, located inside the .xbmc/userdata/ subredditsFile = xbmc.
Database folder. We are concerned with the translatePath("special://profile/
videos DB. It is prefixed with ‘MyVideos’… addon_data/"+addonID+"/subreddits")
this translates into .xbmc/userdata/
addon = xbmcaddon.Addon() def getDbPath(): addon_data/plugin.video.ludent/
filterRating = int(addon. path = xbmc. subreddits
getSetting("filterRating"))
filterVoteThreshold = int(addon.getS translatePath("special://userdata/ def addSubreddit():
etting("filterVoteThreshold")) Database") keyboard = xbmc.Keyboard(‘’,

In order to read settings of type bool you will need files = os.listdir(path) translation(30001))
to do something like: latest = "" keyboard.doModal()
for file in files: if keyboard.isConfirmed() and
filter = addon.getSetting("filter")
== "true" if file[:8] == ‘MyVideos’ keyboard.getText():
and file[-3:] == ‘.db’: subreddit = keyboard.
We are also setting the main URL, plug-in handle
and the user agent for it: if file > latest: getText()
latest = file fh = open(subredditsFile,

return os.path.join(path, ‘a’)
latest) fh.write(subreddit+’\n’)
fh.close()
pluginhandle = int(sys.argv[1]) getPlayCount(url): Once we have the database
urlMain = "http://www.reddit.com" location, we can get the play count using a This function also demonstrates how to take
userAgent = "Mozilla/5.0 (Windows NT simple SQL query. The MyVideo database a text input from the user. Here we are calling
6.2; WOW64; rv:22.0) Gecko/20100101 contains a table called files, which keeps a the Keyboard function with a text title. Once it
Firefox/22.0" record of all the video files played in XBMC by detects the keyboard, it writes the input in the
opener = urllib2.build_opener() filename. In this case it will be URL. subreddits file with a newline character.
opener.addheaders = [(‘User-Agent’,
userAgent)] dbPath = getDbPath() getYoutubeUrl(id): When we locate a YouTube
conn = sqlite3.connect(dbPath) URL to play, we pass it on to the YouTube plug-in
11 Reading localised strings c = conn.cursor() (plugin.video.youtube) to handle the playback. To
As mentioned, XBMC uses strings.xml to do so, we need to call it in a certain format…
serve up the text. In order to read those strings, def getPlayCount(url):
you will need to use getLocalizedString. c.execute(‘SELECT playCount FROM def getYoutubeUrl(id):
url = "plugin://plugin.
translation = addon. files WHERE strFilename=?’, [url])
getLocalizedString result = c.fetchone() video.youtube/?path=/root/
translation(30002) if result: video&action=play_video&videoid=" +
result = result[0] id
In this example, translation(30002) will if result:
return the string "Hot" when it is running in an return int(result) return url
English environment. return 0
return -1

The Python Book 85

Work with Python

Similarly for Vimeo: On the same lines, we can build a function to if spl[i]:
place links as well… subreddit = spl[i]
def getVimeoUrl(id): entries.
url = "plugin://plugin.video. def addLink(name, url, mode,
iconimage, description, date): append(subreddit)
vimeo/?path=/root/video&action=play_ entries.sort()
video&videoid=" + id u = sys.argv[0]+"?url="+urllib. for entry in entries:
quote_plus(url)+"&mode="+str(mode) if entry in defaultEntries:
return url addDir(entry.title(),
ok = True
And for Dailymotion: liz = xbmcgui.ListItem(name, "r/"+entry, ‘listSorting’, "")
iconImage="DefaultVideo.png", else:
def getDailyMotionUrl(id): thumbnailImage=iconimage) addDirR(entry.title(),
url = "plugin://plugin.video. liz.setInfo(type="Video",
infoLabels={"Title": name, "Plot": "r/"+entry, ‘listSorting’, "")
dailymotion_com/?url=" + id + description, "Aired": date}) addDir("[ Vimeo.com ]",
"&mode=playVideo" liz.setProperty(‘IsPlayable’,
‘true’) "domain/vimeo.com", ‘listSorting’,
return url ok = xbmcplugin. "")
addDirectoryItem(handle=int(sys.
Once we have the video URL resolved into the argv[1]), url=u, listitem=liz) addDir("[ Youtu.be ]", "domain/
respective plug-in, playing it is very simple: return ok youtu.be", ‘listSorting’, "")

def playVideo(url): Based on the abstractions we have just created, addDir("[ Youtube.com
listitem = xbmcgui. we can create the base functions which will ]", "domain/youtube.com",
populate the content. But before we do that, ‘listSorting’, "")
ListItem(path=url) let’s first understand how Reddit works. Most of
xbmcplugin. the Reddit content filters are provided through addDir("[ Dailymotion.com
something called Subreddits. This allows you to ]", "domain/dailymotion.com",
setResolvedUrl(pluginhandle, True, view discussions related to a particular topic. In ‘listSorting’, "")
listitem) our plug-in we are interested in showing videos;
we also want to show trailers, documentaries addDir("[B]-
13 Populating plug-in content listing etc. We access these using Subreddits. For "+translation(30001)+" -[/B]", "",
xbmcplugin contains various routines example, for trailers it would be reddit.com/r/ ‘addSubreddit’, "")
for handling the content listing inside the trailers. For domains we can use /domain; for
plug-ins UI. The first step is to create directory example, to get all the YouTube videos posted xbmcplugin.
entries which can be selected from the XBMC on Reddit, we will call reddit.com/domain/ endOfDirectory(pluginhandle)
UI. For this we will use a function called youtube.com. Now you may ask what is the
xbmcplugin.addDirectoryItem. guarantee that this Subreddit will only list Here, the penultimate entry makes a call to
videos? The answer is that it may not. For that addSubreddit. listSorting takes care of sorting
For our convenience we will be abstracting reason we scrape the site ourselves to find out the data based on criteria such as Hot,
addDirectoryItem to suit it to our purpose, so videos. More on this in the next step. New etc. It also calls in Reddit’s JSON function,
that we can set name, URL, mode, icon image which returns nice easy-to-parse JSON data.
and type easily. The first base function we’ll define is index().
This is called when the user starts the plug-in. We have created a settings entry for all the
def addDir(name, url, mode, sorting criteria. Based on what is set, we go
iconimage, type=""): def index(): ahead and build out the sorted list.
defaultEntries = ["videos",
u = sys.argv[0]+"?url="+urllib. def listSorting(subreddit):
quote_plus(url)+"&mode="+str(mode)+" "trailers", "documentaries", if cat_hot:
&type="+str(type) "music"] addDir(translation(30002),

ok = True entries = defaultEntries[:] urlMain+"/"+subreddit+"/hot/.
liz = xbmcgui.ListItem(name, if os.path. json?limit=100", ‘listVideos’, "")
iconImage="DefaultFolder.png", exists(subredditsFile):
thumbnailImage=iconimage) if cat_new:
liz.setInfo(type="Video", fh = open(subredditsFile, addDir(translation(30003),
infoLabels={"Title": name}) ‘r’)
ok = xbmcplugin. urlMain+"/"+subreddit+"/new/.
addDirectoryItem(handle=int(sys. content = fh.read() json?limit=100", ‘listVideos’, "")
argv[1]), url=u, listitem=liz, fh.close()
isFolder=True) spl = content.split(‘\n’) if cat_top_d:
return ok for i in range(0, len(spl),
1): addDir(translation(30004)+":
"+translation(30007),
urlMain+"/"+subreddit+"/
top/.json?limit=100&t=day",
‘listVideos’, "")

xbmcplugin.
endOfDirectory(pluginhandle)

86 The Python Book

Work with Python

def listVideos(url): In the code listed to the left here, we are
currentUrl = url opening the URL, then – based on regular
xbmcplugin.setContent(pluginhandle, "episodes") expression matches – we are discovering
content = opener.open(url).read() the location title, description, date, ups,
spl = content.split(‘"content"’) downs and rating. We are also locating
for i in range(1, len(spl), 1): video thumbnails and then passing them on
entry = spl[i] to XBMC.
try:
match = re.compile(‘"title": "(.+?)"’, re.DOTALL).findall(entry) Later in the code, we also try to match the
title = match[0].replace("&amp;", "&") URL to a video provider. With our plug-in we are
match = re.compile(‘"description": "(.+?)"’, re.DOTALL). supporting YouTube, Vimeo and Dailymotion.
If this is detected successfully, we call the
findall(entry) helper functions to locate the XBMC plug-
description = match[0] in based playback URL. During this whole
match = re.compile(‘"created_utc": (.+?),’, re.DOTALL).findall(entry) parsing process, if any exception is raised, the
downs = int(match[0].replace("}", "")) whole loop is ignored and the next JSON item
rating = int(ups*100/(ups+downs)) is parsed.
if filter and (ups+downs) > filterVoteThreshold and rating <
15 Installing & running the add-on
filterRating: You can install the add-on using one of
continue the following two methods:
• You can copy the plug-in directory to
title = title+" ("+str(rating)+"%)" .xbmc/addons.
match = re.compile(‘"num_comments": (.+?),’, re.DOTALL). • You can install the plug-in from the zip file. To
findall(entry) do so, compress the add-on folder into a zip file
comments = match[0] using the command:
description = dateTime+" | "+str(ups+downs)+" votes: $ zip -r plugin.video.ludent.zip
"+str(rating)+"% Up | "+comments+" comments\n"+description plugin.video.ludent
match = re.compile(‘"thumbnail_url": "(.+?)"’, re.DOTALL).
findall(entry) To install the plug-in from the zip file, open
thumb = match[0] XBMC, go to System then Add-ons, then click
matchYoutube = re.compile(‘"url": "http://www.youtube.com/ ‘Install from zip file’. The benefit of installing
watch\\?v=(.+?)"’, re.DOTALL).findall(entry) from a zip file is that XBMC will automatically
matchVimeo = re.compile(‘"url": "http://vimeo.com/(.+?)"’, try to install all the dependent plug-ins as well.
re.DOTALL).findall(entry)
url = "" Once you have the plug-in installed, you can
if matchYoutube: run it by going to the Videos Add-ons section of
XBMC, selecting Get More… and then clicking
url = getYoutubeUrl(matchYoutube[0]) on LUD Reddit Viewer.
elif matchVimeo:
You can access the settings dialog of the
url = getVimeoUrl(matchVimeo[0].replace("#", "")) plug-in by right-clicking the LUD Reddit Viewer,
if url: then selecting ‘Add-on settings’.

addLink(title, url, ‘playVideo’, thumb, description, date) So, you have seen how robust and powerful
except: XBMC’s extension system is. In this example,
we were able to leverage the full power of
pass Python (including those magical regular
match = re.compile(‘"after": "(.+?)"’, re.DOTALL).findall(entry) expression matches) from within XBMC.
xbmcplugin.endOfDirectory(pluginhandle) XBMC itself also offers a robust UI framework,
if forceViewMode: which provides a very professional look for
our add-on.
xbmc.executebuiltin(‘Container.SetViewMode(‘+viewMode+’)’)
As powerful as it may seem, we have only
14 Populating the episode view (listing videos) built a video plug-in. XBMC’s extension system
At this point we have the URL in hand, which returns JSON data; now we need to extract the also provides a framework for building fully
data out of it which will make sense to us. fledged programs (called Programs). We will
cover this in a later issue.
By looking at the JSON data, you can see there’s a lot of interesting information present here. For
example, url is set to youtube.com/watch?v=n4rTztvVx8E; title is set to ‘The Counselor – Official
Trailer’. There also many other bits of data that we will use, such as ups, downs, num_comments,
thumbnail_url and so on. In order to filter out the data that we need, we will use regular expressions.

There is one more thing to note: since we are not presenting directories any more but are ready to
place content, we have to set the xbmcplugin.setContent to episodes mode.

The Python Book 87

Work with Python Matplotlib
generated output
A simple Python
program for
Polynomial Fitting!

A Python script Finding help
that uses SciPy to is easy
process an image

Scientific computing
with NumPy Powerfulcalculationswith

NumPy, SciPy and Matplotlib

Resources NumPy is the primary Python package for NumPy also works with Pygame, a Python
performing scientific computing. It has a package for creating games, though explaining
NumPy: powerful N-dimensional array object, tools its use is beyond of the scope of this article.
for integrating C/C++ and Fortran code, linear
www.numpy.org algebra, Fourier transform, and random It is considered good practice to try the
number capabilities, among other things. various NumPy commands inside the Python
SciPy: NumPy also supports broadcasting, which is shell before putting them into Python programs.
a clever way for universal functions to deal in
www.scipy.org a meaningful way with inputs that do not have The examples in this article are using either
exactly the same form. Python shell or iPython.
Matplotlib:
Apart from its capabilities, the other 01 Installing NumPy
www.matplotlib.org advantage of NumPy is that it can be integrated Most Linux distributions have a
into Python programs. In other words, you may ready-to-install package you can use. After
get your data from a database, the output of installation, you can find out the NumPy version
another program, an external file or an HTML you are using by executing the following:
page and then process it using NumPy.
$ python
This article will show you how to install Python 2.7.3 (default, Mar 13 2014, 11:03:55)
NumPy, make calculations, plot data, read and [GCC 4.7.2] on linux2
write external files, and it will introduce you to Type "help", "copyright", "credits" or
some Matplotlib and SciPy packages that work "license" for more information.
well with NumPy. >>> numpy.version.version

88 The Python Book

Work with Python

Traceback (most recent call last): 03Making simple
File "<stdin>", line 1, in <module> calculations

NameError: name 'numpy' is not defined
>>> import numpy
>>> numpy.version.version
'1.6.2'
>>>

Not only have you found the NumPy version but
you also know that NumPy is properly installed.

02 AboutNumPy
Despite its simplistic name, NumPy is
a powerful Python package that is mainly for
working with arrays and matrices.

There are many ways to create an array but
the simplest is by using the array() function:

>>> oneD = array([1,2,3,4])

The aforementioned command creates a 04 Using arrays with NumPy 06 Writing to files
one-dimensional array. If you want to create a NumPy not only embraces the indexing Writing variables to a file is largely
two-dimensional array, you can use the array() methods used in typical Python for strings and similar to reading a file. If you have an array
function as follows: lists but also extends them. If you want to select variable named aa1, you can easily save its
a given element from an array, you can use the contents into a file called aa1.txt by using the
>>> twoD = array([ [1,2,3], following notation: following command:
... [3,3,3],
... [-1,-0.5,4], >>> twoD[1,2] In [17]: np.savetxt("aa1.txt", aa1)
... [0,1,0]] )
You can also select a part of an array (a slice) As you can easily imagine, you can read
You can also create arrays with more dimensions. using the following notation: the contents of aa1.txt later by using the
loadtxt() function.
03 Making simple calculations >>> twoD[:1,1:3]
using NumPy 07 Common functions
Given an array named myArray, you can find Finally, you can convert an array into a Python NumPy supports many numerical and
the minimum and maximum values in it by list using the tolist() function. statistical functions. When you apply a function
executing the following commands: to an array, the function is automatically applied
05 Reading files to all array elements.
>>> myArray.min() Imagine that you have just extracted
>>> myArray.max() information from an Apache log file using AWK and When working with matrices, you can find the
you want to process the text file using NumPy. inverse of a matrix AA by typing “AA.I”. You can
Should you wish to find the mean value of all also find its eigenvalues by typing “np.linalg.
array elements, run the next command: The following AWK code finds out the total eigvals(AA)” and its eigenvector by typing “np.
number of requests per hour: linalg.eig(BB)”.

>>> myArray.mean() $ cat access.log | cut -d[ -f2 | cut -d] 08 Working with matrices
-f1 | awk -F: '{print $2}' | sort -n | uniq A special subtype of a two-dimensional
Similarly, you can find the median of the array -c | awk '{print $2, $1}' > timeN.txt NumPy array is a matrix. A matrix is like an
by running the following command: array except that matrix multiplication replaces
The format of the text file (timeN.txt) with the element-by-element multiplication. Matrices
>>> median(myArray) data is the following: are generated using the matrix (or mat) function
as follows:
The median value of a set is an element that 00 191
divides the data set into two subsets (left 01 225 In [2]: AA = np.mat('0 1 1; 1 1 1; 1 1 1')
and right subsets) with the same number of 02 121
elements. If the data set has an odd number of 03 104 You can add two matrices named AA and BB by
elements, then the median is part of the data typing AA + BB. Similarly, you can multiply them
set. On the other side, if the data set has an Reading the timeN.txt file and assigning it to a by typing AA * BB.
even number of elements, then the median is new array variable can be done as follows:
the mean value of the two centre elements of
the sorted data set. aa = np.loadtxt("timeN.txt")

The Python Book 89

Work with Python

09Plotting with 09 Plotting with Matplotlib
Matplotlib The first move you should make is to
install Matplotlib. As you can see, Matplotlib has
many dependencies that you should also install.

The first thing you will learn is how to
plot a polynomial function. The necessary
commands for plotting the 3x^2-x+1
polynomial are the following:

import numpy as np
import matplotlib.pyplot as plt
myPoly = np.poly1d(np.array([3, -1, 1]).
astype(float))
x = np.linspace(-5, 5, 100)
y = myPoly(x)
plt.xlabel('x values')
plt.ylabel('f(x) values')
xticks = np.arange(-5, 5, 10)
yticks = np.arange(0, 100, 10)
plt.xticks(xticks)
plt.yticks(yticks)
plt.grid(True)
plt.plot(x,y)

SciPy is built on top of NumPy The variable that holds the polynomial
and is more advanced is myPoly. The range of values that will
be plotted for x is defined using “x =
In [36]: from scipy.stats import poisson, lognorm Fig 01 np.linspace(-5, 5, 100)”. The other important
In [37]: mySh = 10; variable is y, which calculates and holds the
In [38]: myMu = 10; values of f(x) for each x value.
In [39]: ln = lognorm(mySh)
In [40]: p = poisson(myMu) It is important that you start ipython using
In [41]: ln.rvs((10,)) the “ipython --pylab=qt” parameters in order
Out[41]: to see the output on your screen. If you are
array([ 9.29393114e-02, 1.15957068e+01, 9.78411983e+01, interested in plotting polynomial functions,
you should experiment more, as NumPy can
8.26370734e-07, 5.64451441e-03, 4.61744055e-09, also calculate the derivatives of a function and
4.98471222e-06, 1.45947948e+02, 9.25502852e-06, plot multiple functions in the same output.
5.87353720e-02])
In [42]: p.rvs((10,)) 10 About SciPy
Out[42]: array([12, 11, 9, 9, 9, 10, 9, 4, 13, 8]) SciPy is built on top of NumPy and
In [43]: ln.pdf(3) is more advanced than NumPy. It supports
Out[43]: 0.013218067177522842 numerical integration, optimisations, signal
processing, image and audio processing,
and statistics. The example in Fig. 01 (to the left)
uses a small part of the scipy.stats package that
is about statistics.

The example uses two statistics distributions
and may be difficult to understand even if you
know mathematics, but it is presented in order
to give you a better taste of SciPy commands.

11 Using SciPy for image processing
Now we will show you how to process
and transform a PNG image using SciPy.

The most important part of the code is the
following line:

90 The Python Book

Work with Python

image = np.array(Image.open('SA.png'). Process and transform a PNG
convert('L')) image using SciPy

This line allows you to read a usual PNG 11Using SciPy for
file and convert it into a NumPy array for image processing
additional processing. The program will
also separate the output into four parts and
displays a different image for each of these
four parts.

12 Other useful functions
It is very useful to be able to find out
the data type of the elements in an array; it
can be done using the dtype() function.

Similarly, the ndim() function returns the
number of dimensions of an array.

When reading data from external files, you
can save their data columns into separate
variables using the following way:

In [10]: aa1,aa2 = np.loadtxt("timeN.txt",
usecols=(0,1), unpack=True)

The aforementioned command saves column
1 into variable aa1 and column 2 into variable
aa2. The “unpack=True” allows the data to be
assigned to two different variables. Please
note that the numbering of columns starts
with 0.

13 Fitting to polynomials 13 Fitting to
The NumPy polyfit() function tries to fit Polynomials
a set of data points to a polynomial. The data
was found from the timeN.txt file, created
earlier in this article.

The Python script uses a fifth degree
polynomial, but if you want to use a different
degree instead then you only have to change
the following line:

coefficients = np.polyfit(aa1, aa2, 5)

14 Array broadcasting in NumPy
To close, we will talk more about
array broadcasting because it is a very
useful characteristic. First, you should know
that array broadcasting has a rule: in order
for two arrays to be considered for array
broadcasting, “the size of the trailing axes for
both arrays in an operation must either be the
same size or one of them must be one.”

Put simply, array broadcasting allows
NumPy to “change” the dimensions of an array
by filling it with data in order to be able to do
calculations with another array. Nevertheless,
you cannot stretch both dimensions of an
array to do your job.

The Python Book 91

Work with Python

Each message The server notifies
has a time stamp all clients when a
prefixed to it new client joins

A client can detect Similarly, the server
when the server notifies all clients
exits without
crashing or hanging when a client leaves

Instant messaging with Python

How to program both the client, complete with a GUI, and
server of a simple instant messenger in Python

Resources He’re we’ll be implementing an instant • We’ll also be using threading, which allows a
messenger in Python with a client-server program to do multiple things at once.
A computer – running your favourite Linux architecture. This means each client connects
to the server, which relays any message that • We’ll cover the basics of writing a simple
distribution one client sends to all other clients. The server graphical user interface with GTK, as well as
will also notify the other clients when someone how to interact with that from a different thread.
Internet connection –toaccess joins or leaves the server. The instant messenger
can work anywhere a TCP socket can: on the • Finally, we’ll be touching on the use of
documentation same computer with the loopback interface, regular expressions to easily analyse and extract
across various computers on a LAN, or even data from strings.
Python 2.x, PyGTK and GObject – over the internet if you were to configure your
router correctly. However, our messages aren’t Before getting started, you’ll need to have
packages installed encrypted, so we wouldn’t recommend that. a Python2.x interpreter installed, as well as
the PyGTK bindings and the Python2 GObject
Writing an instant messenger is an interesting bindings. The chances are that if you have a
technical problem that covers a bunch of system with a fair amount of software on it,
areas that you may not have come across while you will already have these packages, so it may
programming before: be easier to wait and see if you’re missing any
libraries when you attempt to import them. All of
• We’ll be employing sockets, which are used the above packages are commonly used, so you
to transmit data across networks. should be able to install them using your distro’s
package manager.

92 The Python Book

Work with Python

01 Theserver import socket Useful
The server will do the following jobs: import re documentation
• Listen for new clients import signal
• Notify all clients when a new client joins import sys Threading: docs.python.org/2/library/
• Notify all clients when a client leaves import time
• Receive and deliver messages to all clients threading.html
03 The Server class
We’re going to write the server side of the The Server class is the main class of our Sockets: docs.python.org/2/library/
instant messenger first, as the client requires instant messenger server. The initialiser of this
it. There will be two code files, so it’s a good class accepts a port number to start listening socket.html
idea to make a folder to keep them inside. You for clients on. It then creates a socket, binds the
can create an empty file with the command socket to the specified port on all interfaces, Regular expressions: docs.python.
touch [filename], and mark that file as and then starts to listen on that port. You can
executable using chmod +x [filename]. This optionally include an IP address in the tuple that org/2/library/re.html
file is now ready to edit in your favourite editor. contains the port. Passing in a blank string like
we have done causes it to listen on all interfaces. The signal handler: docs.python.org/
[liam@liam-laptop Python]$ mkdir The value of 1 passed to the listen function
Python-IM specifies the maximum number of queued 2/library/signal.html
[liam@liam-laptop Python]$ cd connections we can accept. This shouldn’t be
Python-IM/ a problem as we’re not expecting a bunch of PyGTK: www.pygtk.org/
[liam@liam-laptop Python-IM]$ touch clients to connect at exactly the same time.
IM-Server.py pygtk2reference
[liam@liam-laptop Python-IM]$ chmod Now that we have a socket, we’ll create an
+x IM-Server.py empty array that will be later used to store a GObject: www.pygtk.org/
collection of client sockets that we can echo
02 Starting off messages to. The final part is to tell the signal pygtk2reference/gobject-functions.html
As usual, we need to start off with the library to run the self.signal_handler function,
line that tells the program loader what it needs which we have yet to write, when a SIGINT or sockets and then starts an instance of the
to interpret the rest of the file with. In your SIGTERM is sent to the application so that we ClientListener class, which we have yet to
advisor’s case, that line is: can tidy up nicely. write, in a new thread. Sometimes, defining
interfaces you are going to call before you’ve
#!/usr/bin/env python2. class Server(): written them is good, because it can give an
On your system, it may need to be changed to def __init__(self, port): overview of how the program will work without
#!/usr/bin/env/ python2.6 or #!/usr/ worrying about the details.
bin/env python2.7 # Create a socket and bind it to a
port Note that we’re printing information as we go
After that, we’ve written a short comment about along, to make debugging easier should we need
what the application does, and imported the self.listener = socket. to do it. Sleeping at the end of the loop is useful
required libraries. We’ve already mentioned socket(socket.AF_INET, socket.SOCK_ to make sure the while loop can’t run quickly
what the threading and socket libraries are STREAM) enough to hang the machine. However, this is
for. The re library is used for searching strings unlikely to happen as the line that accepts new
with regular expressions. The signal library is self.listener.bind((‘’, connections is blocking, which means that the
used for dealing with signals that will kill the port)) program waits for a connection before moving
program, such as SIGINT. SIGINT is sent when on from that line. For this reason, we need to
Ctrl+C is pressed. We handle these signals so self.listener.listen(1) enclose the line in a try block, so that we can
that the program can tell the clients that it’s print “Listening on port catch the socket error and exit when we can no
exiting rather than dying unexpectedly. The sys {0}”.format(port) longer accept connections. This will usually be
library is used to exit the program. Finally, the # Used to store all of the client when we’ve closed the socket during the process
time library is used to put a sensible limit on how sockets we have, for echoing of quitting the program.
frequently the body of while loops execute. to them
self.client_sockets = [] def run(self):
#!/usr/bin/env python2 # Run the function self.signal_ while True:
# The server side of an instant handler when Ctrl+C is pressed
messaging application. Written as signal.signal(signal.SIGINT, # Listen for clients, and create a
part of a Linux User & Developer self.signal_handler) ClientThread for each new client
tutorial by Liam Fraser in 2013. signal.signal(signal.
import threading SIGTERM, self.signal_handler) print “Listening for
more clients”
04 The server’s main loop
The server’s main loop essentially try:
accepts new connections from clients, (client_socket,
adds that client’s socket to the collection of
client_address) = self.listener.
accept()

except socket.error:
sys.exit(“Could not

The Python Book 93

Work with Python

accept any more connections”) # Remove the specified socket from the the buffer to use while receiving data.
client_sockets list
self.client_sockets. def run(self):
append(client_socket) self.client_sockets. # The thread's loop to receive and
remove(socket) process messages
print “Starting client
thread for {0}”.format(client_ def signal_handler(self, signal, while self.listening:
address) frame): data = ""
# Run when Ctrl+C is pressed try:
client_thread = data = self.socket.
ClientListener(self, client_socket, print "Tidying up"
client_address) # Stop listening for new connections recv(1024)
except socket.error:
client_thread.start() self.listener.close() "Unable to recieve
# Let each client know we are quitting
time.sleep(0.1) data"
self.echo("QUIT") self.handle_msg(data)
05 The echo function time.sleep(0.1)
We need a function that can be called 07 The client thread
from a client’s thread to echo a message to each The class that is used to deal with each # The while loop has ended
client. This function is pretty simple. The most client inherits the Thread class. This means print "Ending client thread
important part is that sending data to sockets is that the class can be created, then started with
in a try block, which means that we can handle client_thread.start(). At this point, the for {0}".format(self.address)
the exception if the operation fails, rather than code in the run function of the class will be run in
having the program crash. the background and the main loop of the Server 09 Tidyingup
class will continue to accept new connections. We need to have a function to tidy up
def echo(self, data): the thread. We’ll call this either when the client
# Send a message to each socket in We have to start by initialising the Thread base sends us a blank string (indicating that it’s
self.client_socket class, using the super keyword. You may have stopped listening on the socket) or sends us the
noticed that when we created a new instance of string “QUIT”. When this happens, we’ll echo to
print "echoing: {0}". the ClientListener class in the server’s main loop, every client that the user has quit.
format(data) we passed through the server’s self variable. We
do this because it’s better for each instance of the def quit(self):
for socket in self.client_ ClientListener class to have its own reference to # Tidy up and end the thread
sockets: the server, rather than using the global one that
# Try and echo to all clients we’ll create later to actually start the application. self.listening = False
self.socket.close()
try: class ClientListener(threading. self.server.remove_
socket.sendall(data) Thread): socket(self.socket)
self.server.echo("{0} has
except socket.error: def __init__(self, server, quit.\n".format(self.username))
print "Unable to send socket, address):
# Initialise the Thread base class 10 Handling messages
message" There are three possible messages our
super(ClientListener, clients can send:
06 Finishing the Server class self).__init__() • QUIT
The remainder of the Server class is # Store the values that have been • USERNAME user
taken up with a couple of simple functions; passed to the constructor • Arbitrary string to be echoed to all clients
one to remove a socket from the collection of
sockets, which doesn’t need an explanation, self.server = server The client will also send a bunch of empty
and the signal_handler function that we talked self.address = address messages if the socket has been closed, so we
about in the initialiser of the class. This function self.socket = socket will end their thread if that happens. The code
stops listening for new connections, and self.listening = True should be pretty self-explanatory apart from
unbinds the socket from the port it was listening self.username = "No the regular expression part. If someone sends
on. Finally, we send a message to each client to Username" the USERNAME message, then the server tells
let them know that we are exiting. The signal will every client that a new user has joined. This is
continue to close the program as expected once 08 The client thread’s loop tested with a regular expression. ^ indicates the
the signal_handler function has ended. The loop that runs in the client thread start of the string, $ indicates the end, and the
is pretty similar to the one in the server. It keeps brackets containing .* extract whatever comes
def remove_socket(self, socket): listening for data while self.listening is true, after “USERNAME ”.
and passes any data it gets to a handle_msg
function that we will write shortly. The value
passed to the socket.recv function is the size of

94 The Python Book

Work with Python

We need to tell GObject that we’ll be class MainWindow(gtk.Window):
using threading def __init__(self):

def handle_msg(self, data): #!/usr/bin/env python2 # Initialise base gtk window class
# Print and then process the message # The client side of an instant super(MainWindow, self).__
we’ve just recieved messaging application. Written as
part of a Linux User & Developer init__()
print "{0} sent: {1}". tutorial by Liam Fraser in 2013. # Create controls
format(self.address, data)
# Use regular expressions to test for import threading self.set_title("IM Client")
a message like "USERNAME liam" import gtk vbox = gtk.VBox()
import gobject hbox = gtk.HBox()
username_result = import socket self.username_label = gtk.
re.search('^USERNAME (.*)$', data) import re Label()
import time self.text_entry = gtk.
if username_result: import datetime Entry()
self.username = send_button = gtk.
# Tell gobject to expect calls from Button("Send")
username_result.group(1) multiple threads self.text_buffer = gtk.
self.server.echo("{0} gobject.threads_init() TextBuffer()
text_view = gtk.
has joined.\n".format(self. 13 The client graphical user interface TextView(self.text_buffer)
username)) The user interface of the client isn’t # Connect events
the main focus of the tutorial, and won’t be self.connect("destroy",
elif data == "QUIT": explained in as much detail as the rest of self.graceful_quit)
# If the client has sent quit then the code. However, the code should be fairly send_button.
close this thread straightforward to read and we have provided connect("clicked", self.send_
links to documentation that will help. message)
self.quit() # Activate event when user presses
elif data == "": Our MainWindow class inherits the gtk Enter
# The socket at the other end is Window class, so we need to start by initialising self.text_entry.
probably closed that using the super keyword. Then we create connect("activate", self.send_
the controls that will go on the window, connect message)
self.quit() any events they have to functions, and finally # Do layout
else: lay out the controls how we want. The destroy vbox.pack_start(text_view)
# It's a normal message so echo it to event is raised when the program is closed, and hbox.pack_start(self.
everyone the other events should be obvious. username_label, expand = False)

self.server.echo(data) GTK uses a packing layout, in which you use
Vboxes and Hboxes to lay out the controls. V
11 Starting the server and H stand for vertical and horizontal. These
The code that actually starts the Server controls essentially let you split a window
class is as follows. Note that you are probably up almost like a table, and will automatically
best picking a high-numbered port as you need decide the size of the controls depending on the
to be root to open ports <1024. size of the application.

if __name__ == "__main__": GTK doesn’t come with a control to enter
# Start a server on port 59091 basic information, such as the server’s IP
server = Server(59091) address, port and your chosen username, so
server.run() we’ve made a function called ask_for_info,
which creates a message box, adds a text
12 Theclient box to it and then retrieves the results. We’ve
Create a new file for the client as we did done this because it’s simpler and uses less
for the server and open it in your favourite editor. code than creating a new window to accept
The client requires the same imports as the the information.
server, as well as the gtk, gobject and datetime
libraries. One important thing we need to do is to
tell GObject that we’ll be using threading, so we
can call functions from other threads and have
the main window, which is running in the main
GTK thread, update.

Work with Python

hbox.pack_start(self.text_ message_format = question) 15 The remainder of MainWindow
entry) entry = gtk.Entry() The rest of the MainWindow class has
entry.show() plenty of comments to explain itself, as follows.
hbox.pack_end(send_button, dialog.vbox.pack_end(entry) One thing to note is that when a client sends a
expand = False) response = dialog.run() message, it doesn’t display it in the text view
response_text = entry. straight away. The server is going to echo the
vbox.pack_end(hbox, expand message to each client, so the client simply
= False) get_text() displays its own message when the server
# Show ourselves dialog.destroy() echoes it back. This means that you can tell if
if response == gtk.RESPONSE_ the server is not receiving your messages when
self.add(vbox) you don’t see a message that you send.
self.show_all() OK:
# Go through the configuration return response_text def add_text(self, new_text):
process # Add text to the text view
self.configure() else:
def ask_for_info(self, return None text_with_timestamp = "{0}
question): {1}".format(datetime.datetime.now(),
# Shows a message box with a text 14 Configuring the client
entry and returns the response This code is run after we’ve added the new_text)
dialog = gtk. controls to the main window, and asks the user # Get the position of the end of
MessageDialog(parent = self, type = for input. Currently, the application will exit if the the text buffer, so we know where to
gtk.MESSAGE_QUESTION, user enters an incorrect server address or port; insert new text
but this isn’t a production system, so that’s fine.
flags = gtk.DIALOG_MODAL | end_itr = self.text_buffer.
def configure(self): get_end_iter()
gtk.DIALOG_DESTROY_WITH_PARENT, # Performs the steps to connect to # Add new text at the end of the buffer
the server
buttons = gtk.BUTTONS_OK_CANCEL, # Show a dialog box asking for server self.text_buffer.insert(end_
address followed by a port itr, text_with_timestamp)

server = self.ask_for_ def send_message(self, widget):
info("server_address:port") # Clear the text entry and send the
# Regex that crudely matches an IP message to the server
address and a port number # We don't need to display it as it
will be echoed back to each client,
regex = re.search('^(\d+\.\ including us.
d+\.\d+\.\d+):(\d+)$', server)
new_text = self.text_entry.
address = regex.group(1). get_text()
strip()
self.text_entry.set_text("")
port = regex.group(2). message = "{0} says: {1}\n".
strip() format(self.username, new_text)
# Ask for a username self.network.send(message)
def graceful_quit(self, widget):
self.username = self.ask_ # When the application is closed,
for_info("username") tell GTK to quit, then tell the
server we are quitting and tidy up
self.username_label.set_ the network
text(self.username) gtk.main_quit()
# Attempt to connect to the server self.network.send("QUIT")
and then start listening self.network.tidy_up()

self.network =
Networking(self, self.username,
address, int(port))

self.network.listen()

The server is going to echo the
message to each client

Work with Python

16 The client’s Networking class difference is that we want to add some things to you’ve started the server, open an instance of
Much of the client’s Networking class is the text view of our window. We do this by using the client and enter 127.0.0.1:port, where
similar to that of the server’s. One difference is the idle_add function of GObject. This allows ‘port’ is the port you decided to use. The server
that the class doesn’t inherit the Thread class – us to call a function that will update the window will print the port it’s listening on to make this
we just start one of its functions as a thread. running in the main thread when it is not busy. easy. Then enter a username and click OK. Here
is an example output from the server with two
class Networking(): def send(self, message): clients. You can use the client over a network
def __init__(self, window, # Send a message to the server by replacing 127.0.0.1 with the IP address of the
server. You may have to let the port through your
username, server, port): print "Sending: {0}". computer’s firewall if it’s not working.
# Set up the networking class format(message)
[liam@liam-laptop Python]$ ./IM-
self.window = window try: Server.py
self.socket = socket. self.socket. Listening on port 59091
socket(socket.AF_INET, socket.SOCK_ Listening for more clients
STREAM) sendall(message) Starting client thread for
self.socket.connect((server, except socket.error: ('127.0.0.1', 38726)
port)) print "Unable to send ('127.0.0.1', 38726) sent: USERNAME
self.listening = True client1
# Tell the server that a new user message" echoing: client1 has joined.
has joined Listening for more clients
self.send("USERNAME {0}". def tidy_up(self): Starting client thread for
format(username)) # We'll be tidying up if either we are ('127.0.0.1', 38739)
quitting or the server is quitting ('127.0.0.1', 38739) sent: USERNAME
def listener(self): client2
# A function run as a thread that self.listening = False echoing: client2 has joined.
listens for new messages self.socket.close() Listening for more clients
# We won't see this if it's us ('127.0.0.1', 38739) sent: client2
while self.listening: that's quitting as the window will says: Hi
data = "" be gone shortly echoing: client2 says: Hi
try: gobject.idle_add(self. ('127.0.0.1', 38726) sent: client1
data = self.socket. window.add_text, "Server has says: Hi
quit.\n") echoing: client1 says: Hi
recv(1024) ('127.0.0.1', 38726) sent: QUIT
except socket.error: def handle_msg(self, data): echoing: client1 has quit.
"Unable to recieve if data == "QUIT": Ending client thread for
('127.0.0.1', 38726)
data" # Server is quitting ^CTidying up
self.handle_msg(data) self.tidy_up() echoing: QUIT
Could not accept any more
# Don't need the while loop to be elif data == "": connections
ridiculously fast # Server has probably closed ('127.0.0.1', 38739) sent:
unexpectedly echoing: client2 has quit.
time.sleep(0.1) Ending client thread for
self.tidy_up() ('127.0.0.1', 38739)
17 Running a function as a thread else:
The listener function above will be run # Tell the GTK thread to add some 21 That’sit!
as a thread. This is trivial to do. Enabling the text when it's ready So, it’s not perfect and could be a little
daemon option on the thread means that it will more robust in terms of error handling, but we
die if the main thread unexpectedly ends. gobject.idle_add(self. have a working instant messenger server that
window.add_text, data) can accept multiple clients and relay messages
def listen(self): between them. More importantly, we have
# Start the listening thread 19 Starting the client learned a bunch of new concepts and methods
The main window is started by initialising of working.
self.listen_thread = an instance of the class. Notice that we don’t
threading.Thread(target=self. need to store anything that is returned. We then
listener) start the GTK thread by calling gtk.main().
# Stop the child thread from keeping
the application open if __name__ == "__main__":
# Create an instance of the main
self.listen_thread.daemon = window and start the gtk main loop
True
MainWindow()
self.listen_thread.start() gtk.main()

18 Finishing the Networking class 20 Trying it out
Again, most of this code is similar to You’ll want a few terminals: one to
the code in the server’s Networking class. One start the server, and some to run clients. Once

The Python Book 97

Work with Python

Replace your shell
with Python

Python is a great programming language, but did you know that
it is even capable of replacing your primary shell (command-line
interface)? Here, we explain all…

Resources We all use shell on a daily basis. For most While the Python programming language
of us, shell is the gateway into our Linux may require you to write longer commands to
You will require a version of Python installed on system. For years and even today, Bash has accomplish a task (due to the way Python’s
your system. The good news is you don’t have to been the default shell for Linux. But it is getting a modules are organised), this is not something
do anything to get it installed. Most of the Linux bit long in the tooth. to be particularly concerned about. You can
distributions already ship with either Python 2.6 or easily write aliases to the equivalent of the Bash
Python 2.7 No need to be offended: we still believe Bash command that you intend to replace. Most of
is the best shell out there when compared to the time there will be more than one way to do
some other UNIX shells such as Korn Shell a thing, but you will need to decide which way
(KSH), C Shell (CSH) or even TCSH. works best for you.

This tutorial is not about Bash being Python provides support for executing
incapable, but it is about how to breathe system commands directly (via the os or
completely new life into the shell to do old subprocess module), but where possible we will
things conveniently and new things which were focus on Python-native implementations, as
previously not possible, even by a long shot. So, this allows us to develop portable code.
without further delay, let’s jump in.

98 The Python Book

Work with Python

SECTION 1: Completing basic shell
tasks in Python

1. File management You can easily write aliases to the
equivalent of the Bash command that
The Python module shutil provides support for you intend to replace
file and directory operations. It provides support
for file attributes, directory copying, archiving
etc. Let’s look at some of its important functions.

shutil module path.join(‘~’, ‘.ssh’)) ‘jsc’, ‘DISPLAY’: ‘/tmp/launch-s2LUfa/
copy (src,dst): Copy the src file to >>> make_archive(archive_name, ‘gztar’, org.x:0’, ‘TERM_PROGRAM’: ‘Apple_
the destination directory. In this root_dir) Terminal’, ‘TERM’: ‘xterm-color’,
mode permissions bits are copied but ‘/Users/kunal/ludarchive.tar.gz’ ‘Apple_PubSub_Socket_Render’: ‘/tmp/
metadata is not copied. launch-kDul5P/Render’, ‘VERSIONER_
copy2 (src,dst): Same as copy() but 2. Interfacing operating system & PYTHON_VERSION’: ‘2.7’, ‘SHLVL’:
also copies the metadata. subprocesses ‘1’, ‘SECURITYSESSIONID’: ‘186a5’,
copytree(src, dst[, symlinks=False[, ‘ANDROID_SDK’: ‘/Applications/MOTODEV_
ignore=None]]): This is similar to ‘cp Python provides two modules to interface Studio_For_Android_2.0.0_x86/android_
-r’, it allows you to copy an entire with the OS and to manage processes, called sdk’,’_’: ‘/System/Library/Frameworks/
directory. os and subprocess. These modules allow you Python.framework/Versions/2.7/bin/
ignore_patterns (*patterns): ignore_ to interact with the core operating system python’, ‘TERM_SESSION_ID’: ‘ACFE2492-
patterns is an interesting function shell and let you work with the environment, BB5C-418E-8D4F-84E9CF63B506’, ‘SSH_
that can be used as a callable for processes, users and file descriptors. AUTH_SOCK’: ‘/tmp/launch-dj6Mk4/
copytree(), it allows you to ignore Listeners’, ‘SHELL’: ‘/bin/bash’,
files and directories specified by the The subprocess module was introduced to ‘TMPDIR’: ‘/var/folders/6s/pgknm8b118
glob-style patterns. support better management of subprocesses 737mb8psz8x4z80000gn/T/’, ‘LSCOLORS’:
rmtree(path[, ignore_errors[, (part of which already exists in the os ‘ExFxCxDxBxegedabagacad’, ‘CLICOLOR’:
onerror]]): rmtree() is used to delete module) in Python and is aimed to replace ‘1’, ‘__CF_USER_TEXT_ENCODING’:
an entire directory. os.system, os.spawn*, os.popen, popen2.* and ‘0x1F5:0:0’, ‘PWD’: ‘/Users/kunaldeo’,
move(src,dst): Similar to mv command it commands.* modules. ‘COMMAND_MODE’: ‘unix2003’}
allows you to recessively move a file
or directory to a new location. os module You can also find out the value for an
environ: environment represents the OS environment value:
Example: environment variables in a string object. >>> os.environ[‘HOME’]
>>>from shutil import copytree, ignore_ ‘/Users/kunaldeo’
patterns example:
>>>copytree(source, destination, >>> import os putenv(varname,value) : Adds or sets
ignore=ignore_patterns(‘*.pyc’, >>> os.environ an environment variable with the given
‘tmp*’)) {‘VERSIONER_PYTHON_PREFER_32_BIT’: variable name and value.
‘no’, ‘LC_CTYPE’: ‘UTF-8’, ‘TERM_
make_archive(base_name, format[, root_ PROGRAM_VERSION’: ‘297’, ‘LOGNAME’: getuid() : Return the current process’s
dir[, base_dir[, verbose[, dry_run[, ‘kunaldeo’, ‘USER’: ‘kunaldeo’, ‘PATH’: user id.
owner[, group[, logger]]]]]]] : Think ‘/System/Library/Frameworks/Python. getlogin() : Returns the username of
of this as a replacement for tar, zip, framework/Versions/2.7/bin:/Users/ currently logged in user
bzip etc. make_archive() creates an kunaldeo/narwhal/bin:/opt/local/sbin:/ getpid(pid) : Returns the process group
archive file in the given format usr/local/bin:/usr/bin:/bin:/usr/sbin:/ id of given pid. When used without
such as zip, bztar, tar , gztar. sbin:/usr/local/bin:/usr/X11/bin:/opt/ any parameters it simply returns the
Archive support can be extended via local/bin:/Applications/MOTODEV_Studio_ current process id.
Python modules. For_Android_2.0.0_x86/android_sdk/ getcwd() : Return the path of the
tools:/Applications/MOTODEV_Studio_For_ current working directory.
Example Android_2.0.0_x86/android_sdk/platform- chdir(path) : Change the current
>>> from shutil import make_archive tools:/Volumes/CyanogenModWorkspace/ working directory to the given path.
>>> import os bin’, ‘HOME’: ‘/Users/kunaldeo’,
>>> archive_name = os.path. ‘PS1’: ‘\\[\\e[0;32m\\]\\u\\[\\e[m\\]
expanduser(os.path.join(‘~’, \\[\\e[1;34m\\]\\w\\[\\e[m\\] \\
‘ludarchive’)) [\\e[1;32m\\]\\$\\[\\e[m\\] \\
>>> root_dir = os.path.expanduser(os. [\\e[1;37m\\]’, ‘NARWHAL_ENGINE’:

The Python Book 99

Work with Python

listdir(path) : Similar to ls, returns command with arguments. On process ready-made, you are in luck. IPython provides a
a list with the content of directories completion it returns the returncode powerful and interactive Python shell which you
and file available on the given path. attribute. can use as your primary shell. IPython supports
Python 2.6 to 2.7 and 3.1 to 3.2 . It supports
two type of Python shells: Terminal based and
Qt based.

Example: Example: Just to reiterate, IPython is purely implemented
>>> os.listdir(“/home/homer”) in Python and provides a 100% Python-
[‘.gnome2’, ‘.pulse’, ‘.gconf’, >>> import subprocess compliant shell interface, so everything you
‘.gconfd’, ‘.beagle’, ‘.gnome2_ have learnt in section 1 can be run inside
private’, ‘.gksu.lock’, ‘Public’, >>> print subprocess.call([“ls”,”-l”]) IPython without any problems.
‘.ICEauthority’, ‘.bash_history’,
‘.compiz’, ‘.gvfs’, ‘.update- total 3684688
notifier’, ‘.cache’, ‘Desktop’,
‘Videos’, ‘.profile’, ‘.config’, drwx------+ 5 kunaldeo staff
‘.esd_auth’, ‘.viminfo’, ‘.sudo_
as_admin_successful’, ‘mbox’, 170 Aug 19 01:37 Desktop
‘.xsession-errors’, ‘.bashrc’, ‘Music’,
‘.dbus’, ‘.local’, ‘.gstreamer-0.10’, drwx------+ 10 kunaldeo staff IPython is already available in most Linux
‘Documents’, ‘.gtk-bookmarks’, distributions. Search your distro’s repositories to
‘Downloads’, ‘Pictures’, ‘.pulse- 340 Jul 26 08:30 Documents look for it. In case you are not able to find it, you
cookie’, ‘.nautilus’, ‘examples. can also install it using easy_install or PyPI.
desktop’, ‘Templates’, ‘.bash_logout’] drwx------+ 50 kunaldeo staff

mkdir(path[, mode]) : Creates a 1700 Aug 19 12:50 Downloads
directory with the given path with the
numeric code mode. The default mode is drwx------@ 127 kunaldeo staff
0777.
makedirs(path[, mode]) : Creates given 4318 Aug 19 01:43 Dropbox IPython provides a lot of interesting features
path (inclusive of all its directories) which makes it a great shell replacement…
recursively. The default mode is 0777. drwx------@ 42 kunaldeo staff
:
1428 Aug 12 15:17 Library
Example:
>>> import os drwx------@ 3 kunaldeo staff Tab completion: Tab completion provides an
>>> path = “/home/kunal/greatdir” excellent way to explore any Python object that
>>> os.makedirs( path, 0755 ); 102 Jul 3 23:23 Movies you are working with. It also helps you to avoid
making typos.
rename (old,new) : The file or drwx------+ 4 kunaldeo staff
directory “old” is renamed to “new”
If “new” is a directory, an error 136 Jul 6 08:32 Music
will be raised. On Unix and Linux, if
“new” exists and is a file, it will drwx------+ 5 kunaldeo staff
be replaced silently if the user has
permission to do so. 170 Aug 12 11:26 Pictures Example :
renames (old,new) : Similar to rename
but also creates any directories drwxr-xr-x+ 5 kunaldeo staff In [3]: import o {hit tab}
recessively if necessary.
rmdir(path) : Remove directory from the 170 Jul 3 23:23 Public objc opcode operator
path mentioned. If the path already
has files you will need to use shutil. -rwxr-xr-x 1 kunaldeo staff optparse os os2emxpath
rmdtree()
1886555648 Aug 16 21:02 androidsdk.tar
subprocess:
drwxr-xr-x 5 kunaldeo staff In [3]: import os
call(*popenargs, **kwargs) : Runs the
170 Aug 16 21:05 sdk

drwxr-xr-x 19 kunaldeo staff In [4]: os.p {hit tab}
os.pardir os.pathconf_names
646 Aug 19 01:47 src os.popen os.popen4
os.path os.pathsep
-rw-r--r-- 1 root staff os.popen2 os.putenv
os.pathconf os.pipe
367 Aug 16 20:36 umbrella0.log os.popen3

STD_INPUT_HANDLE: The standard input Built In Object Explorer: You can add
device. Initially, this is the console input buffer. ‘?’ after any Python object to view
STD_OUTPUT_HANDLE: The standard output its details such as Type, Base Class,
device. Initially, this is the active console String Form, Namespace, File and
screen buffer. Docstring.
STD_ERROR_HANDLE: The standard error
device. Initially, this is the active console Example:
screen buffer.
In [28]: os.path?
SECTION 2: IPython: a ready-made
Python system shell replacement

Type: module

In section 1 we have introduced you to the Base Class: <type ‘module’>
Python modules which allow you to do system
shell-related tasks very easily using vanilla String Form:<module ‘posixpath’ from
Python. Using the same features, you can build
a fully featured shell and remove a lot of Python ‘/System/Library/Frameworks/Python.
boilerplate code along the way. However, if
you are kind of person who wants everything framework/Versions/2.7/lib/python2.7/

posixpath.pyc’>

Namespace: Interactive

File: /System/Library/Frameworks/

100 The Python Book