Software Design Spring 2008 For today you should have: 1) prepared for a quiz 2) worked on your project For next time you should: 1) Write a design proposal a) clean up, amend your project description b) read the project document to see what should be added 2) Read anything in today's notes that we don't cover in class and try out the examples. 3) Install Pyro (Python Remote Objects) On Ubuntu: sudo apt-get install pyro On Fedora: yum install Pyro Download wb/sd/code/RemoteObject.py and run it. Exam next Thursday. Gather/scatter dictionaries --------------------------- Last time we saw gather/scatter tuples. In a parameter list, the ** operator gathers the keyword arguments: def foo(**kwds): print kwds foo(a=1, b=2, name='allen') In an argument list, the ** operator scatters a dictionary: d = dict(a=1, b=2, name='allen') foo(d) # error... foo(**d) # works! I use this frequently to pass options to Tkinter: In pack_demo.py: options = dict(side=LEFT, padx=10, pady=10) b1 = g.bu(text='OK', **options) b2 = g.bu(text='Cancel', **options) b3 = g.bu(text='Help', **options) Override/underride ------------------ The widget wrapper functions in Gui.py use ** to pass around options: # canvas def ca(self, *args, **options): underride(options, fill=BOTH, expand=1) return self.widget(GuiCanvas, *args, **options) What is underride all about? Here is the definition: def underride(d, **kwds): """Add kwds to the dictionary only if they are not already set""" for key, val in kwds.iteritems(): if key not in d: d[key] = val def override(d, **kwds): """Add kwds to the dictionary even if they are already set""" d.update(kwds) The factory pattern ------------------- While we are in the neighborhood, you might notice that all of the widget wrapper classes call Gui.widget and pass a constructor as an argument: def widget(self, constructor, *args, **options): """this is the mother of all widget constructors. the constructor argument is the function that will be called to build the new widget. args is rolled into options, and then options is split into widget option, pack options and grid options """ options.update(dict(zip(Gui.argnames,args))) widopt, packopt, gridopt = split_options(options) # make the widget and either pack or grid it widget = constructor(self.frame, **widopt) if hasattr(self.frame, 'gridding'): self.grid(widget, **gridopt) else: widget.pack(**packopt) return widget According to the Gang of Four, the Factory pattern "provides an interface for creating families of related or dependent objects without specifying their concrete classes (at compile time)" See http://en.wikipedia.org/wiki/Factory_method_pattern Network programming ------------------- 1) low-level communication: sockets 2) medium-level protocol modules 3) high-level distributed programming: remote objects Sockets ------- A socket is a software object that allows a process to connect to a port. A process is a running program. There might be more than one process running the same program. A port is a mailbox that is visible from other computers. Clients and servers ------------------- A common way to organize a distributed computation is with different processes that act as clients and servers. 1) The Web is a client-server system Web browsers are clients They connect to servers, as specified by URLs Servers wait for connections, receive requests, look up files, send replies. Many servers, many clients. Most servers run all the time. Clients are intermittent. 2) A (relatively) simple example: An echo server waits for connections, receives data, "echos" the data back to the client. One server, one client (at a time). ### echo_server.py from socket import * HOST = '' # empty string means the local host PORT = 50007 # arbitrary non-privileged port lsock = socket(AF_INET, SOCK_STREAM) lsock.bind((HOST, PORT)) lsock.listen(1) csock, addr = lsock.accept() print 'Server connected to:', csock.getpeername() while 1: data = csock.recv(1024) if not data: break print 'Server received', `data` csock.send(data) csock.close() lsock.close() ### echo_client.py from socket import * HOST = 'rocky.olin.edu' # the remote host PORT = 50007 # the same port used by the server csock = socket(AF_INET, SOCK_STREAM) csock.connect((HOST, PORT)) print 'Client connected to: ', csock.getpeername() csock.send('Hello, world') data = csock.recv(1024) csock.close() print 'Client received', `data` 1) port numbers: some "well-known" numbers are reserved for particular functions others are available for transient use 2) bind: associate a socket with a particular port 3) listen: make this socket into a (passive) listening socket servers use this method; clients don't 4) accept: wait for a client to connect, and return a tuple lsock is the listening socket csock is the connected socket addr is the address of the client 5) send/recv: can only send and receive strings! 6) connect: clients invoke connect to specify the HOST and PORT of the server Lab Exercise ------------ 1) Find a partner. 2) Boot linux and make sure you have a working network. 3) One of you download echo_client.py from the usual place; the other should download echo_server.py 4) If you have the server, run "hostname" to get the name of your laptop and tell it to your partner. 5) If you have the client, replace the value of HOST with the name of your partner's laptop. 6) Run the server. 7) Run the client. If things go according to plan, the client should say Client connected to: ('10.8.11.90', 50007) Client received 'Hello, world' And the server should say Server connected to: ('10.8.11.90', 36807) Server received 'Hello, world' Notice that the server uses TWO sockets, one for listening and one for send/recv with clients. If it doesn't work, you might have to make a hole in your firewall. 1) Select System->Administration->Firewall 2) Type root password when asked Open "Other ports" and add port=50007, protocol=tcp Or just disable the firewall temporarily. Pros and Cons of sockets ------------------------ +) versatile and powerful -) complicated -) can only send and receive strings! +) can implement any protocol -) have to implement any protocol Protocol modules ---------------- There are built-in (or readily-available) modules that work with many common protocols. For example, urllib 1) parses URLs into (protocol, server, file) 2) opens a socket and connects to the server 3) sends a legal HTTP GET request 4) reads and parses the response 5) returns an object that can be used to access the response Documentation at www.python.org/doc/current/lib/module-urllib.html Example: from urllib import * conn = urlopen('http://ece.olin.edu/sd/code/World.py') print conn for line in conn.fp: print line, OR print conn.read() As another example, HTMLParser reads HTML, processes the syntax, and calls your handler functions for different tags. from HTMLParser import HTMLParser from urllib import * class MyHTMLParser(HTMLParser): def handle_starttag(self, tag, attrs): print "Encountered the beginning of a %s tag" % tag def handle_endtag(self, tag): print "Encountered the end of a %s tag" % tag p = MyHTMLParser() conn = urlopen('http://wb/sd/index.html') for line in conn.fp: p.feed(line) Pros and cons of protocol modules --------------------------------- +) easy to work with existing protocols +) useful tools for both the server and client end -) not as versatile as sockets Remote objects -------------- A little more cutting edge, not in standard modules. But I found something pretty good in the Vaults: Pyro (Python Remote Objects). 1) the server a) creates an object that will be available remotely b) advertises the object on a name server 2) the client a) contacts the name server to find a remote object b) creates a local object that acts as a PROXY for the remote object c) invokes methods on the proxy 3) the middleware (Pyro) when the client invokes a method on a proxy, Pyro a) packages the arguments and transfers them to the server b) invokes the method on the remote object c) packages the return value and transfers it to the client This kind of distributed programming is called Remote Procedure Call (RPC) in procedural languages, and Remote Method Invocation (RMI) in object-oriented languages. Lab Exercise 9 Solutions (sd_hw09_soln.py) ------------------------ 1) IntVar and StringVar If you are working with checkbuttons and radio buttons, you have two choices: a) associate a Tkinter Var object with the widget(s) Tk handles all the interactions, both ways! b) use another kind of variable and handle the interactions yourself Here's an example using IntVar and StringVar: class MyTurtle(Turtle): def __init__(self, world, delay=0.5): Animal.__init__(self, world) self.r = 5 self.heading = 0 self.pen = IntVar() self.pen.set(1) self.color = StringVar() self.color.set('red') self.delay = delay self.draw() world.register(self) And then in the TurtleControl panel: w.row() w.cb(text='Pen', variable=self.turtle.pen) w.gr(2) colors = 'red', 'orange', 'yellow', 'green', 'blue', 'violet' for color in colors: w.rb(anchor=W, text=color, command=self.turtle.redraw, variable=self.turtle.color, value=color) w.endgr() w.endrow() 2) One more example of a Callable in the TurtleControl panel: w.gr(2) w.la(text='Line Width') b1 = w.bu(text='up') w.en_w = w.en(width=5, text='1') b2 = w.bu(text='down') b1.configure(command=Callable(incr_entry, w.en_w, 1)) b2.configure(command=Callable(incr_entry, w.en_w, -1)) w.endgr() When widgets refer to each other, the order of creation can be tricky. In this case I had to 1) create the buttons and the entry 2) configure the buttons to refer to the entry But at least I could use the same function for both buttons: def incr_entry(entry, val): old = int(entry.get()) new = old + val entry.delete(0, END) entry.insert(END, str(new)) mod_python ---------- The following instructions have been tested on Ubuntu, they may require fiddling to work on Fedora. 1) On the machine that will be your server, install the packages: apache2 apache2.2-common libapache2-mod-python libapache2-mod-python-doc debian-helper-scripts Confirm that you are now running a web server by loading the address of the server in a web browser; for example: http://10.8.11.90/ Or run "hostname" to get the name of your server and then paste it into your browser. You should get the default page. Read it; it contains useful information. If not, try running "service apache2 start" as root. 2) Configure apache2 to run Python handlers: As root, edit /etc/apache2/apache2.conf and put the following stanza at the end. AddHandler python-program .py PythonHandler myscript Whenever you edit apache2.conf, you have to tell the running server to reload its config file. As root, run "service apache2 reload" 3) Create a project home page. Go to /var/www. This is the default location where apache looks for your files. There should be one file, index.html, there. As root, create a new directory for your project, and make yourself the owner: mkdir myproject chown downey myproject As yourself, cd into /var/www/myproject and create a file named index.html that contains a minimal web page. Confirm that you can load this page in your browser. http://10.8.11.90/myproject/ 4) In /var/www/myproject, create a file called myscript.py with the following code: from mod_python import apache def handler(req): req.content_type = "text/plain" req.send_http_header() req.write("Hello World!") return apache.OK Confirm that you can execute this code by loading http://10.8.11.90/myproject/myscript.py 5) Now follow the instructions in the mod_python tutorial http://www.modpython.org/live/mod_python-2.7.8/doc-html/tut-what-it-do.html For you, /mywebdir is /var/www/myproject