From andy47@halfcooked.com Sat May 4 03:56:36 2002 Received: from c007.snv.cp.net (h012.c007.snv.cp.net [209.228.33.240]) by hydrogen.sabren.com (8.11.6/8.11.6) with SMTP id g447uZx03377 for ; Sat, 4 May 2002 03:56:35 -0400 Received: (cpmta 2065 invoked from network); 4 May 2002 00:56:19 -0700 Received: from 203.173.133.163 (HELO halfcooked.com) by smtp.halfcooked.com (209.228.33.240) with SMTP; 4 May 2002 00:56:19 -0700 X-Sent: 4 May 2002 07:56:19 GMT Message-ID: <3CD393F4.2060809@halfcooked.com> Date: Sat, 04 May 2002 17:55:32 +1000 From: Andy Todd Organization: Half Cooked Solutions User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0rc1) Gecko/20020417 X-Accept-Language: en-us, en MIME-Version: 1.0 To: workshop@cornerhost.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [workshop] Idiots guide to CGI? Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Hi all, I've only just signed up to the list so I'm sorry if I am repeating anything that has already been discussed. I just need a starter on setting up CGI on my account at cornerhost. I've got the worlds simplest CGI script written in Python but I don't know where to put it or what file suffix I need to give it, etc. I'm quite happy with static html and I can program in Python, I just need a little Apache/CGI help. Thanks in advance. Regards, Andy -- ---------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com From michal@sabren.com Sat May 4 23:31:38 2002 Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g453Vcx06990; Sat, 4 May 2002 23:31:38 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020505033132.LLEL9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com>; Sun, 5 May 2002 03:31:32 +0000 Date: Sat, 4 May 2002 23:28:43 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com cc: workshop-lite@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020505033132.LLEL9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] lesson 00: "hello %s!" Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: ############################################################ ## Hey all, ## ## I've got friends in town this week, so I'm off to a slow ## start here. :) I'm new at this, so let me know if I'm ## moving too fast/slow/etc with this tutorial... ## ## Anyway, here goes! ## ## -Michal ############################################################ ## lesson 00: "hello, %s!" ################################# * Hello, World! The traditonal introduction to a computer language is the "Hello, World!" program. In python, it's a one-liner: ### print "hello, world!" ### Easy, huh? If you load up a python interpreter and type that line, it'll print "hello, world!" to the screen. * The Response Object Web apps don't print their output to a screen, but send it across the internet. For this reason (and others I'll go into later on), we redirect the output to a response object. In the framework that I use, this object is called RES: ### print >> RES, "hello, world!" ### That's it! Our first web application! So how can we see it work? To run this on a cornerhost server, put it in a text file with the extension ".app". Call this one "hello.app" and upload it to your account. Now load it in your browser as if it were a normal web page. If all goes well, you should see: hello, world! If you get an error message, double check that you have the filename correct and that the file contains no leading tabs or spaces - python has very strict rules about whitespace. * Hello, NAME! Let's personalize our greeting a bit. Python lets us insert a variable (a defined value) into our output by using the percent sign: ### name = "Orville" print >> RES, "hello, %s!" % name ### If you upload this script and load it in the browser, the result should be: hello, Orville! * The Request Object Of course, most of the people using your app won't be named Orville. One way of passing a name into our app would be to include it in the query string: http:// ... /hello.app?name=Gomer We can read the query string with the REQ (Request) object: ### name = REQ.get("name") ### Better yet, we can provide a default: ### name = REQ.get("name", "whoever you are") print >> RES, "hello, %s!" % name ### Try running this with and without a "?name=whatever" appended to the url. * Adding a Form But why settle for a default, when we can simply ask users what their names are? This next example uses the REQ.has_key() method to tell if a name has been passed in. If so, it says hello. If not, it shows an HTML form: ### if REQ.has_key("name"): print >> RES, "hello, %s!" % REQ.get("name") else: # notice the single quotes, to prevent conflict with HTML: print >> RES, '
' print >> RES, 'What is your name?' print >> RES, '' print >> RES, '' print >> RES, '
' ### * Yuck! There's nothing wrong with that code, but it sure isn't pretty. If you wanted to make the form look nicer, you'd have to add a whole lot more of those "print >> RES" lines, and that would get old real fast. Worse, it's hard to read and hard to maintain. In the next lesson, we'll look at a way of solving these problems by separating the logic of an app from its presentation. ----- (c)2002 sabren enterprises inc feel free to forward this to a friend! archives at: http://webAppWorkshop.com/ discussion list: http://www.cornerhost.com/mailman/listinfo/workshop lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite From dgeiser13@hotmail.com Sun May 5 10:28:22 2002 Received: from hotmail.com (oe34.law12.hotmail.com [64.4.18.91]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g45ESMx14475 for ; Sun, 5 May 2002 10:28:22 -0400 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sun, 5 May 2002 07:28:16 -0700 X-Originating-IP: [63.52.237.155] From: "Dan Geiser" To: References: <20020505033132.LLEL9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: Re: [workshop] lesson 00: "hello %s!" Date: Sun, 5 May 2002 10:28:00 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MIMEOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Message-ID: X-OriginalArrivalTime: 05 May 2002 14:28:16.0706 (UTC) FILETIME=[18ABFA20:01C1F441] Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: OK, this question might seem a little "after the fact" but how does one get startes setting up Python so we can actually take advantages of these lessons which Michal is going to teach us? Thanks, Dan mailto:dgeiser13@hotmail.com ----- Original Message ----- From: "Michal Wallace" To: Cc: Sent: Saturday, May 04, 2002 11:28 PM Subject: [workshop] lesson 00: "hello %s!" > ############################################################ > ## Hey all, > ## > ## I've got friends in town this week, so I'm off to a slow > ## start here. :) I'm new at this, so let me know if I'm > ## moving too fast/slow/etc with this tutorial... > ## > ## Anyway, here goes! > ## > ## -Michal > ############################################################ > > ## lesson 00: "hello, %s!" ################################# > > * Hello, World! > > The traditonal introduction to a computer language is the > "Hello, World!" program. In python, it's a one-liner: > > ### > print "hello, world!" > ### > > Easy, huh? If you load up a python interpreter and type > that line, it'll print "hello, world!" to the screen. > > * The Response Object > > Web apps don't print their output to a screen, but send it across the > internet. For this reason (and others I'll go into later on), we > redirect the output to a response object. In the framework that I use, > this object is called RES: > > ### > print >> RES, "hello, world!" > ### > > > That's it! Our first web application! > > So how can we see it work? > > To run this on a cornerhost server, put it in a text file with the > extension ".app". Call this one "hello.app" and upload it to your > account. > > Now load it in your browser as if it were a normal web page. If all > goes well, you should see: > > hello, world! > > > If you get an error message, double check that you have the filename > correct and that the file contains no leading tabs or spaces - python > has very strict rules about whitespace. > > * Hello, NAME! > > Let's personalize our greeting a bit. Python lets us insert a variable > (a defined value) into our output by using the percent sign: > > ### > name = "Orville" > print >> RES, "hello, %s!" % name > ### > > If you upload this script and load it in the browser, the > result should be: > > > hello, Orville! > > * The Request Object > > Of course, most of the people using your app won't be named > Orville. One way of passing a name into our app would be to include it > in the query string: > > > http:// ... /hello.app?name=Gomer > > > We can read the query string with the REQ (Request) object: > > ### > name = REQ.get("name") > ### > > Better yet, we can provide a default: > > ### > name = REQ.get("name", "whoever you are") > print >> RES, "hello, %s!" % name > ### > > Try running this with and without a "?name=whatever" > appended to the url. > > * Adding a Form > > But why settle for a default, when we can simply ask users what their > names are? This next example uses the REQ.has_key() method to tell if > a name has been passed in. If so, it says hello. If not, it shows an > HTML form: > > ### > if REQ.has_key("name"): > print >> RES, "hello, %s!" % REQ.get("name") > else: > # notice the single quotes, to prevent conflict with HTML: > print >> RES, '
' > print >> RES, 'What is your name?' > print >> RES, '' > print >> RES, '' > print >> RES, '
' > ### > > * Yuck! > > There's nothing wrong with that code, but it sure isn't pretty. If > you wanted to make the form look nicer, you'd have to add a whole lot > more of those "print >> RES" lines, and that would get old real fast. > Worse, it's hard to read and hard to maintain. > > In the next lesson, we'll look at a way of solving these problems by > separating the logic of an app from its presentation. > > > ----- > > (c)2002 sabren enterprises inc > feel free to forward this to a friend! > > archives at: http://webAppWorkshop.com/ > discussion list: http://www.cornerhost.com/mailman/listinfo/workshop > lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite > > > _______________________________________________ > workshop mailing list > workshop@cornerhost.com > http://www.cornerhost.com/mailman/listinfo/workshop > From michal@sabren.com Sun May 5 15:17:04 2002 Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g45JH3x15290 for ; Sun, 5 May 2002 15:17:04 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020505191658.EBSV9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> for ; Sun, 5 May 2002 19:16:58 +0000 Date: Sun, 5 May 2002 15:14:03 -0400 (Eastern Daylight Time) From: Michal Wallace Subject: Re[2]: [workshop] lesson 00: "hello %s!" To: workshop@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE References: <20020505033132.LLEL9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> In-Reply-To: X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020505191658.EBSV9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: On Sun, 5 May 2002 10:28:00 -0400, Dan Geiser wrote: > OK, this question might seem a little "after the fact" but how does one > get startes setting up Python so we can actually take advantages of these > lessons which Michal is going to teach us? Well, you can upload a file to your account with the extension ".app" and try it out there. It's already set up. To get python for yourself, go to http://www.python.org/ I haven't packaged up my web framework in quite a while. I'll be doing that soon. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From dgeiser13@hotmail.com Tue May 7 11:43:31 2002 Received: from hotmail.com (oe21.law12.hotmail.com [64.4.18.125]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g47FhVx23507 for ; Tue, 7 May 2002 11:43:31 -0400 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 7 May 2002 08:43:25 -0700 X-Originating-IP: [63.53.13.52] From: "Dan Geiser" To: References: <20020505033132.LLEL9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> <20020505191658.EBSV9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: Re: Re[2]: [workshop] lesson 00: "hello %s!" Date: Tue, 7 May 2002 11:43:24 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Message-ID: X-OriginalArrivalTime: 07 May 2002 15:43:25.0712 (UTC) FILETIME=[ED133100:01C1F5DD] Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: So I can download Python and just try out these exercises directly on my own machine? Thanks, Dan ----- Original Message ----- From: "Michal Wallace" To: Sent: Sunday, May 05, 2002 3:14 PM Subject: Re[2]: [workshop] lesson 00: "hello %s!" > On Sun, 5 May 2002 10:28:00 -0400, Dan Geiser wrote: > > > OK, this question might seem a little "after the fact" but how does one > > get startes setting up Python so we can actually take advantages of these > > lessons which Michal is going to teach us? > > > Well, you can upload a file to your account with the > extension ".app" and try it out there. It's already set up. > > To get python for yourself, go to http://www.python.org/ > > I haven't packaged up my web framework in quite a while. > I'll be doing that soon. > > Sincerely, > > Michal J Wallace > Sabren Enterprises, Inc. > ------------------------------------- > contact: michal@sabren.com > hosting: http://www.cornerhost.com/ > my site: http://www.sabren.net/ > -------------------------------------- > > > _______________________________________________ > workshop mailing list > workshop@cornerhost.com > http://www.cornerhost.com/mailman/listinfo/workshop > From andy47@halfcooked.com Tue May 7 18:44:55 2002 Received: from smtp3.ihug.com.au (mail@smtp3.ihug.com.au [203.109.250.76]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g47Misx24878 for ; Tue, 7 May 2002 18:44:54 -0400 Received: from p58-max20.syd.ihug.com.au (halfcooked.com) [203.173.153.186] by smtp3.ihug.com.au with esmtp (Exim 3.22 #1 (Debian)) id 175DhQ-0005tW-00; Wed, 08 May 2002 08:44:48 +1000 Message-ID: <3CD858C3.9060203@halfcooked.com> Date: Wed, 08 May 2002 08:44:19 +1000 From: Andy Todd Organization: Half Cooked Solutions User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0rc1) Gecko/20020417 X-Accept-Language: en-us, en MIME-Version: 1.0 To: workshop@cornerhost.com Subject: Re: [workshop] lesson 00: "hello %s!" References: <20020505033132.LLEL9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> <20020505191658.EBSV9799.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Dan Geiser wrote: > So I can download Python and just try out these exercises directly on my own > machine? > > Thanks, > Dan > > ----- Original Message ----- > From: "Michal Wallace" > To: > Sent: Sunday, May 05, 2002 3:14 PM > Subject: Re[2]: [workshop] lesson 00: "hello %s!" > > > >>On Sun, 5 May 2002 10:28:00 -0400, Dan Geiser wrote: >> >> >>>OK, this question might seem a little "after the fact" but how does one >>>get startes setting up Python so we can actually take advantages of >> > these > >>>lessons which Michal is going to teach us? >> >> >>Well, you can upload a file to your account with the >>extension ".app" and try it out there. It's already set up. >> >>To get python for yourself, go to http://www.python.org/ >> >>I haven't packaged up my web framework in quite a while. >>I'll be doing that soon. >> >>Sincerely, >> >>Michal J Wallace >>Sabren Enterprises, Inc. >>------------------------------------- >>contact: michal@sabren.com >>hosting: http://www.cornerhost.com/ >>my site: http://www.sabren.net/ >>-------------------------------------- >> > Err, no. The example scripts that Michal is posting need his web framework (http://weblib.sourceforge.net/) to work properly. As he mentioned in another post (http://www.cornerhost.com/mailman/private/innercircle/2002-April/000233.html) the documentation is a little basic and the CVS (source code) is a little out of date. He has promised to package the latest code, but until thats available you are limited to just Python on any local machine. Mind you, thats pretty rich and good enough for most of us ;-) It also wouldn't hurt to walk through one of the Python tutorials in parallel so you get a feeling for the full scope of the language as well. For the new to programming the recommendation is usually Alan Gaulds introduction at http://www.freenetpages.co.uk/hp/alan.gauld/, for the more experienced (although it is suitable for beginners as well) try Dive into Python (http://diveintopython.org/) written by Mark Pilgrim and hosted at cornerhost. Regards, Andy -- ---------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com From michal@sabren.com Wed May 8 16:05:08 2002 Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g48K57x29507; Wed, 8 May 2002 16:05:08 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020508200502.JUTD22408.rwcrmhc53.attbi.com@c-24-98-91-150.atl.client2.attbi.com>; Wed, 8 May 2002 20:05:02 +0000 Date: Wed, 8 May 2002 16:02:33 -0400 (Eastern Daylight Time) From: Michal Wallace To: innercircle@cornerhost.com cc: workshop@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020508200502.JUTD22408.rwcrmhc53.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] generic sql storage in python Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: (crossposted to innnercircle and workshop lists) I'll be covering this in detail in an upcoming webAppWorkshop lesson, but if anyone's interested you might want to check out the storage module: http://cvs.sabren.com/sixthdev/cvsweb.cgi/storage/Storage.py?rev=1.1.1.1 http://cvs.sabren.com/sixthdev/cvsweb.cgi/storage/MySQLStorage.py?rev=1.1.1.1 Give it a database connection in the constructor, it'll let you fetch and save individual records as python dictionaries. The only requirement is that you have a key named "ID"... It's a fairly simple piece of code. Storage is the generic case, MySQLStorage is for MySQL. There's also a MockStorage which works like an in-memory database (probably only useful for testing purposes): http://cvs.sabren.com/sixthdev/cvsweb.cgi/storage/MockStorage.py?rev=1.1.1.1 Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From andy47@halfcooked.com Thu May 9 02:10:04 2002 Received: from new-smtp2.ihug.com.au (mail@new-smtp2.ihug.com.au [203.109.250.28]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g496A3x31418; Thu, 9 May 2002 02:10:03 -0400 Received: from p60-max20.syd.ihug.com.au (halfcooked.com) [203.173.153.188] by new-smtp2.ihug.com.au with esmtp (Exim 3.22 #1 (Debian)) id 175h7n-0006OY-00; Thu, 09 May 2002 16:10:00 +1000 Message-ID: <3CDA1297.6090801@halfcooked.com> Date: Thu, 09 May 2002 16:09:27 +1000 From: Andy Todd Organization: Half Cooked Solutions User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0rc1) Gecko/20020417 X-Accept-Language: en-us, en MIME-Version: 1.0 To: workshop@cornerhost.com CC: innercircle@cornerhost.com Subject: Re: [workshop] generic sql storage in python References: <20020508200502.JUTD22408.rwcrmhc53.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Michal Wallace wrote: > (crossposted to innnercircle and workshop lists) > > I'll be covering this in detail in an upcoming > webAppWorkshop lesson, but if anyone's interested > you might want to check out the storage module: > > http://cvs.sabren.com/sixthdev/cvsweb.cgi/storage/Storage.py?rev=1.1.1.1 > http://cvs.sabren.com/sixthdev/cvsweb.cgi/storage/MySQLStorage.py?rev=1.1.1.1 > > Give it a database connection in the constructor, > it'll let you fetch and save individual records as > python dictionaries. The only requirement is that > you have a key named "ID"... > > It's a fairly simple piece of code. Storage is the > generic case, MySQLStorage is for MySQL. > > There's also a MockStorage which works like an in-memory > database (probably only useful for testing purposes): > > http://cvs.sabren.com/sixthdev/cvsweb.cgi/storage/MockStorage.py?rev=1.1.1.1 > > > Sincerely, > > Michal J Wallace > Sabren Enterprises, Inc. > ------------------------------------- > contact: michal@sabren.com > hosting: http://www.cornerhost.com/ > my site: http://www.sabren.net/ > -------------------------------------- > Got it, tried it, loved it. Just a couple of points though ;-) 1. There is a bug (I think) in MySQLStorage.py. To make it work I had to add an extra line to the _update method (at line 48 in the CVS file), making it look like; def _update(self, table, **row): sql = "UPDATE " + table + " SET" for col,val in row.iteritems(): sql += " " + col + "='" + str(val) + "'," sql = sql[:-1] sql += " WHERE ID = %d" % row["ID"] self.cur.execute(sql) return self.fetch(table, row["ID"] Without the "WHERE" clause it always trys to update every row in the table. 2. A trap for young players, when Michal said 'The only requirement is that you have a key named "ID"...' I didn't realise the full implications of what he said. I just created a column called 'ID'. This perplexed me for quite a while (which isn't that hard) until I realised that this column HAD to be the primary key *and* had to be an AUTO_INCREMENT column, e.g.; CREATE TABLE test_storage ( ID INTEGER AUTO_INCREMENT, ..., PRIMARY KEY (ID) ); Where ... is your other column definitions. Still it got me reading the manual to find out what on earth cursor._insert_id returned so thats a good thing as far as I'm concerned. It also took me a couple of passes to get my syntax right, when using the 'store' method, you need to pass the data in as a series of parameters, e.g.; >>> db = MySQLdb.connect( db='test' ) >>> storage = MySQLStorage( db ) >>> storage.store('test_storage', COL1='COL1 Value', COL2='Col2 Value') Don't - as I did - try and pass in a dictionary of column name:value pairs because that just confuses everyone. Regards, Andy -- ---------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com From michal@sabren.com Thu May 9 03:26:02 2002 Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g497Q2x31738; Thu, 9 May 2002 03:26:02 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020509072556.EJYF25765.rwcrmhc54.attbi.com@c-24-98-91-150.atl.client2.attbi.com>; Thu, 9 May 2002 07:25:56 +0000 Date: Thu, 9 May 2002 03:23:26 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com, workshop-lite@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020509072556.EJYF25765.rwcrmhc54.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] lesson 01: the spec Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: # lesson 01: The Spec * Jumping In I changed my mind. Instead of talking theory for the first ten lessons, let's start with a real app. My current project is a control panel for cornerhost. The idea is to give customers more power over their accounts, and cut back on the number of routine support tasks I have to handle personally. I'll try to focus on the technology and the development process, and gloss over the control-panel-specific details. Still, it's important to know where you're going, so let's talk a bit about how the app will be laid out. * N Little Tiers When I build a web app, it usually has several tiers: - a SQL ("sequel") database - a reusable object-relational mapper - an object model - the program logic - HTML (or XML) templates - CSS, Javascript, and other client-side stuff Sometimes, you have more or fewer tiers, so it's called n-tier for short. Here are two things to remember about n-tier applications: - A tier should talk only to the tiers directly above or below it. (For example, - Don't blur the edges. (For example, keep SQL out of your python, and keep python out of your HTML) It's often extra work to follow these principles, but if I ignore them (eg, writing a quick script with everything in one file) I almost always have to go back and clean up. The control panel, we'll be dealing with each of these tiers. We'll look at each in detail later. For now, let's consider the requirements. * Expect to be Wrong I like to design my apps up front, so I know where I'm going, but I also know that as I work, I'll keep coming up with better and better ideas. So I'm happy with a rough design. Basically, I want to know: - What should the app do? - What's the simplest way to make it work? Whatever answers we come up with, we'll probably be wrong about some things. That's okay. If we're good, we'll make sure our software is easy to change as we go along. That means we can get started with a very rough idea. So.. On to our two questions. * What should the app do? I'll probably add to this list eventually, but here are the most important features of the control panel. (They're most important because they're the most common requests I get): - Add/Edit/Delete websites (domains) - Add/Edit/Delete email aliases - Show billing information (balance, history, etc) - Logfile Analysis (this part's mostly done) Finally, I want it centralized. In other words, there's one control panel and it manages accounts all the hosting machines. So we'll also need a secure way to communicate with those machines. * What's the simplest way to make it work? The front end of the control panel will be your basic web app. There will be a login page, and several forms that ultimately control records in a database. That's the easy part, and using python and a good web framework, it's less work than you might think. Then there's some scripts that generate configuration files for the web, DNS, and email servers. I've already done a lot of the work here: I've been tweaking my command-line python tools for the past few months. Since these scripts require restarting various long-running programs (like web and email servers), it's important to make sure that changes are queued up and done in batches. This will require some sort of periodic scheduler. Finally, to communicate between the control panel and the actual servers, I plan to use a custom XML-RPC server on each machine. (Again, python makes it easy.) * Let's go! As far as software specs go, this was pretty painless. Still, it's enough to get started. In the next lesson, start writing code. ---------- (c)2002 sabren enterprises inc feel free to forward this to a friend! archives at: http://webAppWorkshop.com/ discussion list: http://www.cornerhost.com/mailman/listinfo/workshop lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite From michal@sabren.com Thu May 9 03:45:28 2002 Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g497jSx31822 for ; Thu, 9 May 2002 03:45:28 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020509074523.GTDE25294.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> for ; Thu, 9 May 2002 07:45:23 +0000 Date: Thu, 9 May 2002 03:42:53 -0400 (Eastern Daylight Time) From: Michal Wallace Subject: Re[2]: [workshop] generic sql storage in python To: workshop@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE References: <20020508200502.JUTD22408.rwcrmhc53.attbi.com@c-24-98-91-150.atl.client2.attbi.com> <3CDA1297.6090801@halfcooked.com> In-Reply-To: <3CDA1297.6090801@halfcooked.com> X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020509074523.GTDE25294.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: On Thu, 09 May 2002 16:09:27 +1000, Andy Todd wrote: > sql += " WHERE ID = %d" % row["ID"] ... > Without the "WHERE" clause it always trys to update every row in the > table. D'oh! :) Thanks! I'd been doing all my testing with MockStorage, and just never noticed. > This perplexed me for quite a while (which isn't that hard) until I > realised that this column HAD to be the primary key *and* had to be an > AUTO_INCREMENT column, e.g.; Yep. > >>> db = MySQLdb.connect( db='test' ) > >>> storage = MySQLStorage( db ) > >>> storage.store('test_storage', COL1='COL1 Value', COL2='Col2 Value') > > Don't - as I did - try and pass in a dictionary of column name:value > pairs because that just confuses everyone. If you want to do that, you can use the ** syntax: >>> storage.store('test_storage', **dictionaryOfNameValuePairs) I figured that way you had the choice. :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From ch@smallestspace.com Thu May 9 17:20:58 2002 Received: from scrabble.freeuk.net (scrabble.freeuk.net [212.126.144.6]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g49LKwx01979 for ; Thu, 9 May 2002 17:20:58 -0400 Received: from du-038-0143.access.clara.net ([217.158.30.143]) by scrabble.freeuk.net with esmtp (Exim 3.33 #1) id 175vLL-0004JF-00 for workshop@cornerhost.com; Thu, 09 May 2002 22:20:56 +0100 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.0.4 Date: Thu, 09 May 2002 22:21:08 +0100 From: Carey Hackett To: Message-ID: In-Reply-To: <20020509072556.EJYF25765.rwcrmhc54.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [workshop] Problem with lesson 00: the spec Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: This runs fine in the browser: print >> RES, "hello, world!" This: name = "Orville" print >> RES, "hello, %s!" % name causes the following exception: uncaught exception while running /web/script/disorganic/disorganic.sc.sabren.com/hello.app Traceback (most recent call last): File "/web/lib/weblib/Engine.py", line 137, in execute self._execute(script) File "/web/lib/weblib/Engine.py", line 127, in _execute exec(script, self.globals, self.locals) File "/web/script/disorganic/disorganic.sc.sabren.com/hello.app", line 1 name = "Orville" print >> RES, "hello, %s!" % name ^ SyntaxError: invalid syntax script input: * form: {} * querystring: * cookie: {'': ''} script output: Content-type: text/html ------------------------------------------------------------------------ weblib (c) copyright 2000-2001 Zike Interactive. All rights reserved. As The Fly would say: "Help me!" Carey -- What you do is of little significance. But it is very important that you do it. -- M.K. Gandhi From michal@sabren.com Thu May 9 18:13:37 2002 Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g49MDbx02162 for ; Thu, 9 May 2002 18:13:37 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020509221331.HRXJ10136.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> for ; Thu, 9 May 2002 22:13:31 +0000 Date: Thu, 9 May 2002 18:11:01 -0400 (Eastern Daylight Time) From: Michal Wallace Subject: Re: [workshop] Problem with lesson 00: the spec To: workshop@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE References: In-Reply-To: X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020509221331.HRXJ10136.rwcrmhc51.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: On Thu, 09 May 2002 22:21:08 +0100, Carey Hackett wrote: > File "/web/script/disorganic/disorganic.sc.sabren.com/hello.app", line > 1 > name = "Orville" > print >> RES, "hello, %s!" % name > ^ > SyntaxError: invalid syntax Hmm. This looks like a whitespace error. It's imporant to upload python files in ASCII mode. Otherwise, unix seems to think that there's only one line. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From michal@sabren.com Sat May 11 00:32:51 2002 Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4B4Wpx07866 for ; Sat, 11 May 2002 00:32:51 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020511043245.ZSYB25294.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> for ; Sat, 11 May 2002 04:32:45 +0000 Date: Sat, 11 May 2002 00:30:13 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020511043245.ZSYB25294.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] abstractions Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Someone asked me off-list about the "keep it separate" concept, and whether it was really useful. I thought I'd share the answer here and maybe get some discussion going. (original message edited slightly for privacy): > I agree that being able to change storage from one place is > handy for architecture changes (like moving MySql to PostgreSql or > something and only having to update your database class). However, > the novice in me is crying out, "is all of this abstraction really > necessary?" Well, in a corporate development shop, you often have separate teams of DBA's, developers, and designers - so that's one reason to separate the layers right there. And as you said, the abstraction layer means you can switch databases at a moments notice... But what's the point if you're a one-man show and you don't see any need to ever switch databases? It's the rule of "Once And Only Once". For example, say you decide to use straight SQL, and you need a small app to add, edit, and delete records for each of three tables. What do you do? Hard-code the INSERT, UPDATE, and DELETE statements? That's certainly easiest, but it's also a lot of mindless work. Do you make one HTML form for adding records and a separate one for editing? Actually, you'll probably move a lot faster if you do it this way. You can even cut and paste. But what happens when you need to add a new column? You've got to change it in the database, then update the code in two different places (the INSERT and the UPDATE). If you rename the table, you've got to change the DELETE statement, too. But also... Is it really just a couple SQL statements we're dealing with here? No, you've got error-trapping code to deal with. If you're dealing with strings, you've got to check for single-quotes in the values so they don't mess up your SQL. Plus, somewhere along the line, you've actually got to open the connection and send the data. Why repeat the same code over and over? When you see that happening, it's time to refactor that logic into a function or an object. Here's a link for you, from my blog about 2 years ago. It doesn't really have much to do with the database layer, but it shows how I abstracted out a generic form handler for my objects once I had them, and how it made the code a lot nicer: http://www.sabren.net/rants/2000/05/20000526a.php3 > At a certain point does all of this generic code become too generic? > So generic that it needs to be re-customized slightly for each project? That hasn't been my experience. I do make plenty of changes over time, but it usually means that I'm adding features that now improve all my apps at once. (Assuming I don't break compatability, in which case I don't mind going back to update old apps - but then I have test cases to point out what broke) > Also, isn't it simpler to have your sql right in your code? I think it's simpler to write: db.delete("tablename", id) Rather than: $db->query("DELETE FROM tablename WHERE ID=$id") But that's could just be a matter of taste. > Wouldn't that make it easier to debug instead of trying to dig around > in class files to figure out where an error took place. Maybe, but... - In python you get a nice stack trace so you see exactly where it occurred - You're reusing the same code over and over again, so if an error does occur, you fix it ONCE, and you never have to worry about it again. (Especially if you write a test case for it) Heaven forbid you find an obscure error in the way you did it in one table and realize you've got to go back and fix it in ALL your queries. - You can debug and test it on its own. For example, my storage layer is backed up by test cases, which I can run just by typing a simple command. (Also, for the storage system I posted, the MySQLStorage tests subclass the MockStorage tests, so I didn't have to write extra code) > Another issue I have is with code reuse in general. If you > need to tweak a piece of reused code that is called from > several sites, it can be very dangerous if there is a bug > or something. So start from scratch each time? :) You don't have to use the exact same files. Especially if you're using a good version control system. Also, if you have test cases for your stuff, it's easy to see what broke. (And if something broke and you missed it, you can just write another test case) > For the most part I agree with centralized code and functions ... > Plus I have a tiny library of functions I reuse all the time in all > my sites, but they're just little building blocks that I can stack > together. At a certain point it feels like all the classes exists > purely for their own sake. "There's a class to generalize all the > storage you do in a database which is called by the class that > figures out all of the datatypes in a table. This in turn > is wrapped up in a class for setting up local variables..." and on > and on. It feels like more code than is needed. If I can see the > datamodel SQL file, then I don't need some abstraction that figures > out what's in my tables for me right? --------- I didn't really respond to this below... But I no longer try to figure out what's in the table by reading metadata. I specify it explicitly in my object model, and make sure the object model matches up exactly with the tables in my database. It's much simpler, and it also paves the way for generating your CREATE TABLES and whatnot automatically from the object model -- (though I haven't done that yet). --------- If your abstractions are getting in your way, you need better abstractions. :) The important thing is to simplify the interface. If it's simple to use, it probably doesn't matter how complex the pieces are -- until it comes time to change the pieces anyway :) If it's not simple to use, why did you make it? :) And no, you shouldn't have classes for their own sake. Have classes when you need to encapsulate a certain set of behaviors - especially when you do the same thing over and over again. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From sholden@holdenweb.com Tue May 14 16:05:42 2002 Received: from mail.holdenweb.com (mail.holdenweb.com [64.224.159.178]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4EK5gx30581 for ; Tue, 14 May 2002 16:05:42 -0400 Received: from COMPUTER [64.224.159.178] by mail.holdenweb.com (SMTPD32-6.06) id AE14155D00B0; Tue, 14 May 2002 16:05:40 -0400 Message-ID: <015601c1fb81$f01893a0$7601a8c0@holdenweb.com> From: "Steve Holden" To: Date: Tue, 14 May 2002 16:00:03 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [workshop] Idiots Guide to CGI Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Andy Todd wrote: > I just need a starter on setting up CGI on my account at cornerhost. > I've got the worlds simplest CGI script written in Python but I don't > know where to put it or what file suffix I need to give it, etc. > > I'm quite happy with static html and I can program in Python, I just > need a little Apache/CGI help. Thanks in advance. The things that gave me a little trouble were: a) My particular FTP tool couldn't make the CGI scripts excecutable (sheesh, Windows software...). Michal set the permissions for me before I wrote my Python distribution program (for the site's makefile to drive), and now I have a shell account so I can do what I want. b) I needed to make python files recognised as executable content. I chose to retain the .py extension and so I had to my .htaccess file. If you want to go that way just add an .htaccess file that says: AddHandler cgi-script py However, you'l probably find that *.cgi is already treated as executable (as long as the permissions are correct), and in that case it's just a matter of the right shebang line ... start your scripts with #!/usr/bin/python and all should be well. Anything else? regards Steve ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ ----------------------------------------------------------------------- From michal@sabren.com Fri May 17 17:47:52 2002 Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4HLlqx11791; Fri, 17 May 2002 17:47:52 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020517214747.OYHQ18801.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com>; Fri, 17 May 2002 21:47:47 +0000 Date: Fri, 17 May 2002 17:47:28 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com cc: workshop-lite@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020517214747.OYHQ18801.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] lesson 02: data objects Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: # lesson 02: data objects * A Quick Intro to Objects When I plan a new web app, I start with the data objects - objects whose main purpose is to hold data. What are objects? Instead of boring you with definitions, let's use an example. To begin, fire up a python interpreter. If you don't have python yet, you can download it from http://www.python.org/ Since python's interpreter is interactive, you can follow along with the example by typing them at the ">>>" prompt. * My Dog Spot Let's create a simple class to represent dogs. By convention, we name classes with capital letters: >>> class Dog: pass ... The "..." means that python is waiting for more input. We don't have any more input (that's what "pass" means) so just press enter. That's all it took to create a class called Dog. A Dog class should contain information that applies to dogs. For example, most dogs have four legs: >>> Dog.legs = 4 But suppose we want to talk about a particular dog - an instance of the class Dog: >>> spot = Dog() Now we have a Dog named "spot" (we name instances and other variables in lower case to tell them apart from classes). Since spot is a Dog, he should have four legs. To test that, we can ask python: >>> spot.legs 4 Yep, spot has four legs. But suppose spot has a horrible skiing accident. (Don't worry, this is staged - no animals were hurt in the filming of this workshop.) >>> spot.legs = 2 >>> Dog.legs 4 >>> spot.legs 2 As you can see, python lets us record the fact that while most Dogs have four legs, poor spot has only two. * Spot is a Data Object We said that objects are reusable chunks of data and behavior. The Dog class has a little bit of data (legs=4), but no real behavior. All you can do is instantiate it (make an instance of it). Since a Dog instance's main purpose is to store data, spot is a data object. If we wanted, we could record all kinds of data about spot, just by adding more fields to the Dog class: name, breed, color, and birthday might all make sense. * Who Let the Dogs Out? Let's say we're building a web app for the neighborhood kennel. We're going to use a Dog object to represent each dog. Of course, we can't change our program every time a dog checks in or out. We'd go nuts! Instead, we'll store our data objects in some kind of database. In an ideal world, we'd store objects in an object database. There is at least one object database for python, but I've never used it, so I can't say much about it. ( http://www.amk.ca/zodb/ ) I will say that, in general, object databases are hard to query. Asking for dog # 5 is quick and painless, but asking "how many beagles are in the kennel?" usually requires actual code to loop through and count, and I don't like that. (I hope I'm not spreading FUD here - if I am, please let me know, because I would *love* to work with a really useful object database. LDAP servers seem to have a bit more strength in the object-querying department, but I don't have much experience with LDAP either.) Meanwhile, I use MySQL. MySQL doesn't know anything about objects, but it's fairly easy to write some generic routines to convert between MySQL records and python objects. Plus, MySQL is very fast, and gives us access to SQL (a powerful and somewhat standardized query language). We'll take a closer look at how this object-relational mapping is done in the next lesson. ---------- (c)2002 sabren enterprises inc feel free to forward this to a friend! archives at: http://webAppWorkshop.com/ discussion list: http://www.cornerhost.com/mailman/listinfo/workshop lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite From michal@sabren.com Sat May 18 23:33:33 2002 Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4J3XXx18050 for ; Sat, 18 May 2002 23:33:33 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020519033327.PFAL18801.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> for ; Sun, 19 May 2002 03:33:27 +0000 Date: Sat, 18 May 2002 23:32:26 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020519033327.PFAL18801.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] webAppWiki Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: I've just made a sample app for you all to play with. It's 100% built with things I haven't talked about yet, so maybe it'll actually raise some questions. :) It's the webAppWiki and you can get it at: http://www.webappworkshop.com/apps/webAppWiki-0.1.tgz This is a very simple wiki, in just over 70 lines of python (not counting whitespace and comments). You'll need a cornerhost account to run it, since I haven't packaged up the libraries it uses yet - sorry about that. If you don't have an account, you can see it in action here: http://www.webappworkshop.com/apps/webAppWiki-0.1/wiki.app Here's the source code: http://www.webappworkshop.com/apps/webAppWiki-0.1/wikiApp-source.txt (As I was typing this, I noticed a bug - if you don't end the page with a newline, it cuts off the last character. Anyone want to see if they can fix it?) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From phil@philsown.com Sun May 19 19:18:43 2002 Received: from mercury.sabren.com (mercury.sabren.com [209.61.186.253]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4JNIhx26959 for ; Sun, 19 May 2002 19:18:43 -0400 Received: from HarringtonPC (fl-pbg-2a-3c-174.pbc.adelphia.net [24.49.21.174]) (authenticated (0 bits)) by mercury.sabren.com (8.11.6/8.11.6) with ESMTP id g4JNIhF28885 for ; Sun, 19 May 2002 19:18:43 -0400 Message-ID: <000b01c1ff8b$8a4550a0$ae153118@HarringtonPC> From: "Phillip Harrington" To: References: <20020517214747.OYHQ18801.rwcrmhc52.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: Re: [workshop] lesson 02: data objects Date: Sun, 19 May 2002 19:18:52 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com X-Reply-To: "Phillip Harrington" List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Maybe It's the pain medication, but I died laughing when poor spot had a horrible skiing accident. Thanks for the laughs. Phil ----- Original Message ----- From: "Michal Wallace" To: Cc: Sent: Friday, May 17, 2002 5:47 PM Subject: [workshop] lesson 02: data objects > > # lesson 02: data objects > > * A Quick Intro to Objects > > When I plan a new web app, I start with the data objects - objects > whose main purpose is to hold data. > > What are objects? Instead of boring you with definitions, let's > use an example. > > To begin, fire up a python interpreter. If you don't have > python yet, you can download it from http://www.python.org/ > > Since python's interpreter is interactive, you can follow > along with the example by typing them at the ">>>" prompt. > > > * My Dog Spot > > Let's create a simple class to represent dogs. By convention, > we name classes with capital letters: > > >>> class Dog: pass > ... > > The "..." means that python is waiting for more input. > We don't have any more input (that's what "pass" means) > so just press enter. > > That's all it took to create a class called Dog. > > A Dog class should contain information that applies > to dogs. For example, most dogs have four legs: > > >>> Dog.legs = 4 > > But suppose we want to talk about a particular dog - an > instance of the class Dog: > > >>> spot = Dog() > > Now we have a Dog named "spot" (we name instances and > other variables in lower case to tell them apart from > classes). Since spot is a Dog, he should have four legs. > To test that, we can ask python: > > >>> spot.legs > 4 > > Yep, spot has four legs. > > But suppose spot has a horrible skiing accident. (Don't worry, > this is staged - no animals were hurt in the filming of this > workshop.) > > >>> spot.legs = 2 > >>> Dog.legs > 4 > >>> spot.legs > 2 > > As you can see, python lets us record the fact that while most Dogs > have four legs, poor spot has only two. > > > * Spot is a Data Object > > We said that objects are reusable chunks of data and behavior. The > Dog class has a little bit of data (legs=4), but no real behavior. All > you can do is instantiate it (make an instance of it). > > Since a Dog instance's main purpose is to store data, spot is a data > object. If we wanted, we could record all kinds of data about spot, > just by adding more fields to the Dog class: name, breed, color, and > birthday might all make sense. > > > * Who Let the Dogs Out? > > Let's say we're building a web app for the neighborhood kennel. > We're going to use a Dog object to represent each dog. Of course, > we can't change our program every time a dog checks in or out. > We'd go nuts! Instead, we'll store our data objects in some kind > of database. > > In an ideal world, we'd store objects in an object database. > There is at least one object database for python, but I've > never used it, so I can't say much about it. ( http://www.amk.ca/zodb/ ) > > I will say that, in general, object databases are hard to query. > Asking for dog # 5 is quick and painless, but asking "how many beagles > are in the kennel?" usually requires actual code to loop through and > count, and I don't like that. > > (I hope I'm not spreading FUD here - if I am, please let me know, > because I would *love* to work with a really useful object > database. LDAP servers seem to have a bit more strength in the > object-querying department, but I don't have much experience > with LDAP either.) > > Meanwhile, I use MySQL. MySQL doesn't know anything about objects, but > it's fairly easy to write some generic routines to convert between > MySQL records and python objects. Plus, MySQL is very fast, and gives > us access to SQL (a powerful and somewhat standardized query > language). > > We'll take a closer look at how this object-relational mapping is done > in the next lesson. > > ---------- > > (c)2002 sabren enterprises inc > feel free to forward this to a friend! > > archives at: http://webAppWorkshop.com/ > discussion list: http://www.cornerhost.com/mailman/listinfo/workshop > lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite > > > > _______________________________________________ > workshop mailing list > workshop@cornerhost.com > http://www.cornerhost.com/mailman/listinfo/workshop > From michal@sabren.com Thu May 23 03:14:04 2002 Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4N7E4x08844; Thu, 23 May 2002 03:14:04 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020523071359.DPDB13253.rwcrmhc54.attbi.com@c-24-98-91-150.atl.client2.attbi.com>; Thu, 23 May 2002 07:13:59 +0000 Date: Thu, 23 May 2002 03:10:25 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com, workshop-lite@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020523071359.DPDB13253.rwcrmhc54.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] lesson 03: strongbox Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: # lesson 03: strongbox * In Search of a Data Model Usually, when we're building a web app, we want to deal with more than one type of data object, and store the data in some sort of database. If that's the case, then one of our first goals is to figure out what sort of data objects we're going to work with, and how they relate. A good rule of thumb is that if you have a written description of the system you want to build, and you underline all the nouns, you'll wind up with a list of the data objects you'll need. For example, in the description of the control panel, I mentioned nouns like: - machine - user - domain - email address - website - account balance - billing history All of these can be stored as a single record in a database, so this is a good starting point. [1] * Fleshing it Out On the other hand, this is just a skeleton of a data model. Once we know what the objects are, we want to know what sort of data they'll contain, and how they relate to each other. Let's look at email addresses as an example. When a user adds a domain to her account, I configure the server to accept all mail for that domain. Usually, all the mail goes directly to her inbox. But sometimes, she wants her mail to go to her existing email address, or to forward certain usernames@herdomain to outside addresses. The way I'd handle these fields is to make the catch-all address a property of the domain, and each forwarding address is an object of its own. [2] Here is how I might define the "Domain" and "Forward" data objects in python: ### from strongbox import Strongbox, attr, class Domain(Strongbox): ID = attr(long) userID = attr(long) domain = attr(str) mailto = attr(str) class Forward(Strongbox): ID = attr(long) domainID = attr(long) virtuser = attr(str) mailto = attr(str) ### * What's a Strongbox? Python doesn't usually make it this easy to declare what attributes a class has. Normally, you can add just about anything to an object, and python doesn't care. With a data model, it's nice to be able to restrict which attributes are available, and what can go in them. This is especially important when storing objects in a relational database, because the database only has certain columns. If we try to put data into an attribute without a corresponding column, we'll lose data, so we want to make sure trying would throw an error. The Strongbox module lets us limit an object's attributes in this way. For example, f you put the above code in an *.app file at cornerhost, and then add the line: ### d = Domain(name="webappworkshop.com") ### ... You'll get an error. Why? Because "name" is not a valid attribute of the "Domain" class. Change it to "domain" and it'll work fine. Besides defining simple attributes like these, Strongbox lets you validate attributes according to more advanced python features, such as lambdas, regular expressions, and accessor methods. [3] More importantly, though, defining your data objects in a standard form like this makes it easy to build generic tools to deal with them. For example, if you choose to store these objects in a MySQL database, you can write an class that will take a Strongbox object and a table name, and automatically generate the SQL for creating, retreiving, updating, and deleting the appropriate rows in the database. In fact, I've already done that, and I'll talk about that in an upcoming lesson. In the next lesson, though, we'll take a break from the data objects, and actually build a simple web app! ---- Notes: [1] All the nouns work as objects except "billing history" (which would most likely be a set of "transaction" or "event" objects) We wouldn't store a "website" in the control panel database, either, but we might store the information needed to configure the webserver to serve that site. [2] I treat my data model as a relational database, which I can enhance by adding behaviors. Therefore, I follow the same rules of normalization (eliminating redunandcy) that one normally follows when designing a database. For a quick overview of these rules, see: http://burks.brighton.ac.uk/burks/foldoc/35/28.htm [3] For more info on strongbox's capabilities, see: http://www.sixthdev.com/wiki.cgi/strongbox ----------- (c)2002 sabren enterprises inc feel free to forward this to a friend! archives at: http://webAppWorkshop.com/ discussion list: http://www.cornerhost.com/mailman/listinfo/workshop lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite From michal@sabren.com Mon May 27 08:45:56 2002 Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g4RCjux02667; Mon, 27 May 2002 08:45:56 -0400 Received: from c-24-98-91-150.atl.client2.attbi.com ([24.98.91.150]) by sccrmhc01.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020527124550.SDIX7675.sccrmhc01.attbi.com@c-24-98-91-150.atl.client2.attbi.com>; Mon, 27 May 2002 12:45:50 +0000 Date: Mon, 27 May 2002 08:41:24 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com, workshop-lite@cornerhost.com MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE X-Mailer: Mahogany 0.64.1 'Sparc', running under Windows 98 ( ) Message-Id: <20020527124550.SDIX7675.sccrmhc01.attbi.com@c-24-98-91-150.atl.client2.attbi.com> Subject: [workshop] lesson 04: App and WikiApp Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: # lesson 04: WebAppWiki * Hello Again! Our old hello.app file looked like this: ### print >> RES, "Hello, world!" ### Nice and simple. But here's how I'd *REALLY* write it: ### """ HelloApp: a "hello world" webApp """ from sixthday import App class HelloApp(App): def act_(self): self.write("Hello, world!") if __name__=="__main__": print >> RES, HelloApp(REQ).act() ### * What Does all That Mean? Very quickly, here's what that code says in English: We want to work with App objects, which are defined in a python package called sixthday. A HelloApp is just like an App, except: when a HelloApp is asked to "act_", it says "Hello, world!" If someone's trying to actually run this file (rather than reuse our definitions somewhere else): Make a new HelloApp, passing it the REQuest object for input, and tell the HelloApp to "act". Send whatever it says out to the RESponse object. * WikiApp: A Little More Challenging We've grown from one line of code to six and a comment. What did it buy us? If all we want our program to do is say hello, we've gained nothing. But most programs are more complex. For example, let's consider a simple wiki. A wiki is a website that anybody can edit. It's not easy to explain if you've never used one, so if you haven't, follow this link to the original wiki: http://www.c2.com/cgi/wiki?WelcomeVisitors (And by the way: that particular wiki is FULL of great writing about object-oriented programming and the "eXtreme Programming" practices. It's definitely worth wandering around in!) A wiki has several required features: - show a page, given a WikiPageNameLikeThis - provide a form where anyone can edit a page - save the page once its been edited - WikiPageNames automatically turn into hyperlinks There's a bigger list at: http://www.c2.com/cgi/wiki?WikiPrinciples but I wanted to keep this simple. If you notice, the first three features each occur in a separate HTTP request: Click a link to bring up a page, click a link to bring up the form, click a submit button to post and save. * Apps are Objects! The logic behind those three actions is simple. But how should we organize it? Do we make three separate scripts? One script with a bunch of if-then statements? Both approaches are common. The first few times I built an an app in VBScript (ASP), I went for the "one big file" approach. I found this made ASP files hard to debug. Later I experimented with separate files for each feature; big files full of common functions; separate files for logic and HTML; and a ColdFusion approach called FuseBox where the whole app is one big switch statement, with each feature defined outside the main file. My point is: organizing code becomes a real problem as projects get bigger, and there are lots of opinions out there about what works best. App is mine. The magic of App is that each App can handle multiple functions, and those functions can be accessed dynamically, based on an "action" parameter in the query string or form post. In other words, WikiApp's structure might look like this: ### class WikiApp(App): def act_show(self): pass def act_edit(self): pass def act_save(self): pass print >> RES, WikiApp(REQ).act() ### When the "act()" message is sent on the last line, the WikiApp figures out which method to call based on an "action" parameter. For example, browsing to http://.../wiki.app?action=show would run the act_show() method. If no ?action=XXX is given, Apps call a default method named "act_()", which we saw earlier in the HelloApp. In a wiki, our default action might be to show the front page: ### def act_(self): self.input["page"]="FrontPage" self.act_show() ### What is self.input? Well, in this case, it's just the REQ object. But we call it "self.input" inside the App because Apps don't necessarily have to be run through the web. * Apps Get Around That bears repeating: App objects do not necesarily have to run through the web. Where else might they run? Perhaps in a GUI app, or from the command line, from inside another App, or from an automated test case.[1] Because Apps are objects, they can be used just about anywhere. They can also be subclassed: you can create a BetterWikiApp that is just like WikiApp except for one little part, and you don't even have to cut and paste: ### class BetterWikiApp(WikiApp): ... ### * Check it Out: Anyway, with those concepts out of the way, the rest of the wikiApp code is pretty straightforward. Take a look at the code and see if you can figure out how it works: http://www.webappworkshop.com/apps/webAppWiki-0.1/wikiApp-source.txt You can try it out here: http://www.webappworkshop.com/apps/webAppWiki-0.1/wiki.app And you can download it here: http://www.webappworkshop.com/apps/webAppWiki-0.1.tgz Feel free to post any questions you have to the workshop list! * Next up: Another App! Next time, we'll make a "beginners" attempt at a database-backed App, and see what we can do to make it better. * Notes: [1] Automated test cases make programming *much* more fun, but that's a topic for another day. For more on this idea, see: http://www.c2.com/cgi/wiki?CodeUnitTestFirst ----------- (c)2002 sabren enterprises inc feel free to forward this to a friend! archives at: http://webAppWorkshop.com/ discussion list: http://www.cornerhost.com/mailman/listinfo/workshop lessons only: http://www.cornerhost.com/mailman/listinfo/workshop-lite From michal@sabren.com Mon Sep 23 21:49:52 2002 Received: from imf28bis.bellsouth.net (mail028.mail.bellsouth.net [205.152.58.68]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8O1nq800778; Mon, 23 Sep 2002 21:49:52 -0400 Received: from [192.168.1.102] ([68.154.2.242]) by imf28bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20020924014505.DPVE20600.imf28bis.bellsouth.net@[192.168.1.102]>; Mon, 23 Sep 2002 21:45:05 -0400 Date: Mon, 23 Sep 2002 21:39:50 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Subject: [workshop] rantelope: day one Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: rantelope: day 001 ------------------ Today I started work on a content management system called Rantelope. I will be posting technical discussion of the code to the webAppWorkshop list every day this week, as I work on adding new features. You can try rantelope out interactively as it develops; simply visit the rantelope homepage: http://www.rantelope.com/ After taking a look at the live demo, you should follow the link to the "DAY 001" code, as it will be understand the rest of this article. ==[ today's objective ]== Today's goal was to create a VERY rough weblogging tool. I wanted to allow multiple blogs (called "channels" in rantelope), and multiple entries (called "stories") in each channel. For today, the only output will be RSS. RSS is an XML format designed for syndicating blogs and newsfeeds. ==[ implementation ]== If you look at the DAY 001 code, most of the logic for the application is in the file "rantelope.app" (Click the version number (1.1.1.1) to see the code) As you can see, the code is broken into three parts: - the object model - the interface - the main code, which invokes the program I will talk about each part briefly. I'm going to be pretty technical here. If you don't understand something, chances are taht other people have the same question, so please ask on the webAppWorkshop list! Okay: -- object model -- The object model is pretty simple. As I said, we have Channels and Stories. One Channel can have many Stories. Both have some attributes which correspond to RSS tags. The Channel class has a method to write an RSS file. It uses a template language I wrote called zebra. The "rss.zb" file is a fairly simple example of a zebra template. It simply turns a a Channel object into an RSS feed. Like python, zebra has an indentation based syntax: - Lines starting with an asterisk ("*") are zebra statements. - Items between "{:" and ":}" are variables to be insterted into the output. Hopefully, the code in the rss.zb is self explanatory. -- the interface -- Getting bac to rantelope.app, we see the next part is a big calss called "RantelApp". Since so may web applications involve adding, editing, deleting, and listing things, it pays to make it simple to do these things. Since we are using an object oriented language, we simply can subclass another application and tailor it to our needs. In this case, RantelApp subclasses a class called AdminApp, which contains genericadd/edit/delete and list routines. RantelApp also uses zebra templates. Each call to generic_show() includes a classname (Channel or Story) and a zebra template filename. So, for example, the edit_story() method near the end of the class takes a Story and renders it using the template in the file frm_story.zb ... The logic of determining WHICH Story gets loaded is handled by generic_show(), which RantelApp inherited from AdminApp. Basically, AdminApp figures out what to do from the query string. (If you're curious, try poking around through the live demo, and watching the URL) The other RantelApp methods are basically the same. A couple quick notes: - To avoid the infamous 'user hits submit twice' problem, we always redirect after a form post. So save_channel() and save_story() both have a call to redirect() - save_story() also loads up the corresponding Channel object and asks it to regenerate its rss file. So RSS gets written whenever a story gets saved. (This probably ought to happen in save_channel(), too) --[ the main code ]-- When RantelApp.save_story() loads up the channel, it gets it from self.clerk ... Every AdminApp has a clerk, which is the object responsible for loading stuff in and out of the storage system. In this case, I'm using a MySQL database. The "dbmap" variable explains how each object and relationship gets mapped to my database. The database itself is defined in the file rantelope.sql. Notice that this is the only place a database connection (sqlRantelope.dbc) is mentioned. If we had another kind of Storage class besides MySQLStorage, then the program would work just as well with it, and nothing would have to be changed except the definition of CLERK. Anyway, that's about it. All that's left is to create a RantelApp instance, run it, and send the result to the browser. --[ the end ]-- That's it. This initial version of Rantelope took about two and a half hours to put together. And I'm sure it shows! If you look at it, it's not pretty AT ALL, and there are probably plenty of bugs. But, this is how a web app gets its start. Since most site authors would rather have a blog than a raw RSS feed, the next task will be to generate HTML. Tomorrow, I will add a simple template system that does this, either with zebra or with the more standard XSLT. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From michal@sabren.com Tue Sep 24 23:06:27 2002 Received: from imf12bis.bellsouth.net (mail112.mail.bellsouth.net [205.152.58.52]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8P36R810139; Tue, 24 Sep 2002 23:06:27 -0400 Received: from [192.168.1.102] ([68.154.2.242]) by imf12bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20020925030338.GTBJ969.imf12bis.bellsouth.net@[192.168.1.102]>; Tue, 24 Sep 2002 23:03:38 -0400 Date: Tue, 24 Sep 2002 22:56:09 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Subject: [workshop] rantelope: day 002 Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: rantelope: day 002 ------------------ Today I added an XSLT template system to Rantelope. It wasn't terribly complicated. Most of the work was just brushing up on my XSLT to get a default template working. The new code and live demo are up on: http://www.rantelope.com/ ==[ today's objective ]== Since we already had an RSS feed, I thought it would make sense to generate HTML by transforming the RSS with XSLT. I could have used my own template language (Zebra), but I want Rantelope to be built on standards, not my personal whims. Zebra has hardly any documentation, whereas there are plenty of great resources for learning XSLT. Using XSLT also means we can potentially reuse the templates for RSS feeds generated by *any* blogging tool, which is a nice plus. ==[ implementation ]== Running an XSLT transformation in python is easy, thanks to the 4Suite library: http://4suite.org/index.xhtml I went ahead and installed 4suite on all cornerhost machines in case anyone wants to play around with it. I wasn't sure how to use it, but a quick google search saved the day. Uche Ogbuji, the principal author of 4Suite, has a tutorial for working with XSLT here: http://uche.ogbuji.net:8080/uche.ogbuji.net/tech/akara/pyxml/python-xslt/ Once I read that, invoking a tranformation was easy. If you're dealing with xml stored in python strings, it boils down to something like this: def transform(xml, xsl): """ A simple wrapper for 4XSLT. """ from Ft.Xml.Xslt.Processor import Processor from Ft.Xml.InputSource import DefaultFactory proc = Processor() ## @TODO: these fromString() calls should ## really include a URI... xslObj = DefaultFactory.fromString(xsl) proc.appendStylesheet(xslObj) xmlObj = DefaultFactory.fromString(xml) return proc.run(xmlObj) Well, an RSS feed looks something like this: my channel http://rantelope.com/ my first channel. first post! http://rantelope.com/ rantelope rules! I was missing the tag yesterday, but once I fixed that, it just took a little reading to put together a default XSLT template. It looked something like this: plainXSLT =\ '''\ <xsl:value-of select="rss:title"/>

''' Making that the default involved several steps. First, I had to add a .template field to the Channel class. It uses the "default" option of the strongbox.attr() type: class Channel(Strongbox): # ... template = attr(str, default=plainXSLT) Second, I had to add a "template" field to my table in MySQL. Finally, I updated all the existing records to use the new template: for c in CLERK.match(Channel): c.template = plainXSLT c.writeFiles() CLERK.store(c) The writeFiles() function is also new. Now that I want to write HTML as well as RSS, it made sense to refactor things a bit. So we now have: class Channel(Strongbox): #... def toRSS(self): return zebra.fetch("rss", BoxView(self)) def toHTML(self, input=None): rss = input or self.toRSS() return transform(rss, self.template) def writeFiles(self): rss = self.toRSS() print >> open(self.path + self.rssfile, "w"), rss if self.template: print >> open(self.path + self.htmlfile, "w"), self.toHTML(rss) I did make one other change: I wanted to keep people from changing the output filename to something like "rantelope.app", which could possibly overwrite the actual rantelope application - a HUGE security flaw! :) So I hard-coded the output directory to "./out/" and added a validation rule to the .rssfile and .htmlfile attributes: class Channel(Strongbox): #... rssfile = attr(str, okay=lambda x: "/" not in x and x.endswith(".rss")) htmlfile = attr(str, okay=lambda x: "/" not in x and x.endswith(".html")) The "okay" option also accepts regular expressions, so I might use those in the future instead of these long, ugly lambdas. (Lambdas are a way of writing really short python functions on the fly.) Now if you try and input an invalid filename, you'll get a nasty error screen. Eventually, it'll be a nicely formatted error message, of course. That's pretty much it for today. Tomorrow, I'm going to expand rantelope from just a blogging tool to a true content management system capable of managing an entire website. ----- PS: I tried to make this one a little more "skimmable", and not force people to go hunt down the code... Is this better? Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From michal@sabren.com Fri Sep 27 01:55:00 2002 Received: from imf17bis.bellsouth.net (mail317.mail.bellsouth.net [205.152.58.177]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8R5t0829094; Fri, 27 Sep 2002 01:55:00 -0400 Received: from [192.168.1.102] ([68.154.2.242]) by imf17bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20020927055140.XBLC4612.imf17bis.bellsouth.net@[192.168.1.102]>; Fri, 27 Sep 2002 01:51:40 -0400 Date: Fri, 27 Sep 2002 01:43:42 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Subject: [workshop] rantelope day 003: the channel hierarchy Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: rantelope: day 003 ------------------ Yes, I'm a day behind schedule. But this is the third development day, so it's still day 003. :) Today I implemented Yahoo!-style hierarchical channels. Whether this is a good idea or not remains to be seen. :) As usual, the new code and live demo are up on: http://www.rantelope.com/ ==[ today's objective ]== I promised last time to make rantelope a true content management system, capable of managing an entire website, rather than just a blog. It's not there yet, but it's closer. I wanted hierarchy. I wanted to be able to maintain blogs, normal web pages, link directories, FAQs, and other content all through the same interface. The trick was figuring out what that meant. :) I toyed around with a couple possibilities, and finally decided that my Channel object really ought to be a node in a tree. So, the goal was to have let each Channel have some number of subchannels as well as some number of stories. That means a channel can take on any of several forms: - blog: 1..* stories, 0 subchannels - about page: 1 story, 0 subchannels - link directory: 0..* stories, 0..* subchannels I liked this idea, so that's what I built. ==[ implementation ]== I ran in to a wall almost immediately. My "sixthday" libraries handle relationships between classes transparently: you can have many stories in a channel and the Clerk does all the magic of keeping the relationships up to date in the database. Unfortunately, I haven't gotten around to implementing hierarchical structures like Node, which wants to look like this: class Node: parent = link(Node) kids = linkset(Node) # breadcrumb trail: top/node1/subnode/etc... crumbs = linkset(Node) I can build objects like that just fine; the problem is persistence. Clerk sees the recursive structure and freaks out. I'm going to fix that, but who knows how long that will take? So I wrote a kludge. One nice things about object-oriented languages is that you can encapsulate your messes. I knew how I wanted Node to look to the outside world, but it wouldn't yet work right out of the box. So I hacked together an implementation and then dressed it up to look nice, even though it wasn't. The end result is a file called Node.py, which you can see in today's CVS if you're interested: http://cvs.sabren.com/sixthdev/cvsweb.cgi/rantelope/Node.py?rev=1.1 It's ugly and ought to be replaced, so I won't go into it. But, once Node worked, I could make Channel a Node subclass, so now Channels have "kids" and "parents" and "crumbs". Adding the interface was more or less trivial. When you view a channel, you now see the breadcrumb trail at the top, a list of subchannels in the middle, and a new "add subchannel" link. I did have to introduce a few lines of cruft in the RantelApp.show_channel() command. It's ugly, but I'll show it because it shows the kind of thing that's going on behind those generic_show() calls: # note: strongbox.BoxView is a proxy class that gives # Strongboxen a zebra-template-friendly dict interface: def show_channel(self): chan = self.clerk.fetch(Channel, long(self.input["ID"])) chan.clerk = self.clerk model = {"errors":[]} model.update(BoxView(chan)) model["kids"]= [BoxView(k) for k in chan.kids] model["crumbs"]= [BoxView(k) for k in chan.crumbs] print >> self, zebra.fetch("sho_channel", model) Zebra expects a data model built from dictionaries and lists of dictionaries. To render a zebra template, you just make a dictionary with all the data the template needs, and call zebra.fetch() The only other thing I did was fix a bug with the filenames. Last time, I added code to force the RSS and html filenames to end in ".rss" and ".html". The looked lik this: rssfile = attr(str, okay=lambda x: "/" not in x and x.endswith(".rss")) htmlfile = attr(str, okay=lambda x: "/" not in x and x.endswith(".html")) I hinted that I could use a regular expression instead, and it turns out I had to. Otherwise an empty string wouldn't validate, and I wanted to let people leave these fields blank. It now says: rssfile = attr(str, okay="([^/]+.rss|^$)" ) htmlfile = attr(str, okay="([^/]+.html|^$)" ) ... Which I think is a lot nicer. Anyway, that's day three in a nutshell. Now that we've got the potential for channels within channels, it makes sense to let those channels share templates, so at some point we'll break the XSLT off into its own class. Tomorrow, though, I'm dusting off my python-powered indexer, and giving rantelope a search engine. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From hansv@net4all.be Fri Sep 27 05:22:41 2002 Received: from firebird.planetinternet.be (brussels-smtp.planetinternet.be [195.95.34.12]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8R9Mf830397 for ; Fri, 27 Sep 2002 05:22:41 -0400 Received: from net4all.be (dpc107.staf.planetinternet.be [194.119.237.107]) by firebird.planetinternet.be (Postfix) with ESMTP id BEA80D3E1B for ; Fri, 27 Sep 2002 11:17:28 +0200 (CEST) Date: Fri, 27 Sep 2002 11:17:38 +0200 Subject: Re: [workshop] rantelope day 003: the channel hierarchy Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Hans verschooten To: workshop@cornerhost.com In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.546) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hydrogen.sabren.com id g8R9Mf830397 Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: Hi Michal, love what you're doing! Just one small point do you know that it's impossible to make posts with accented characters (יטא...) Cheers, Hans From michal@sabren.com Fri Sep 27 10:41:40 2002 Received: from imf10bis.bellsouth.net (mail110.mail.bellsouth.net [205.152.58.50]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8REfe832097 for ; Fri, 27 Sep 2002 10:41:40 -0400 Received: from [192.168.1.102] ([67.33.117.68]) by imf10bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20020927143815.YVMM7035.imf10bis.bellsouth.net@[192.168.1.102]> for ; Fri, 27 Sep 2002 10:38:15 -0400 Date: Fri, 27 Sep 2002 10:30:14 -0400 (Eastern Daylight Time) From: Michal Wallace Subject: Re[2]: [workshop] rantelope day 003: the channel hierarchy To: workshop@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1 Content-Transfer-Encoding: 8BIT Content-Disposition: INLINE References: In-Reply-To: Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: On Fri, 27 Sep 2002 11:17:38 +0200 Hans verschooten wrote: > love what you're doing! Just one small point do you know that it's > impossible to make posts with accented characters (יטא...) Hmmm. You're right. :/ Looks like it gets as far as the database and RSS feed, but the 4Xslt "minidom" parser can't cope with it. It's definitely some sort of unicode problem. I'm a complete newbie there. Anyone have any ideas? Thanks Hans! Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From hansv@net4all.be Fri Sep 27 12:18:12 2002 Received: from firebird.planetinternet.be (brussels-smtp.planetinternet.be [195.95.34.12]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8RGIC832749 for ; Fri, 27 Sep 2002 12:18:12 -0400 Received: from net4all.be (dpc37.staf.planetinternet.be [194.119.237.37]) by firebird.planetinternet.be (Postfix) with ESMTP id C69AFD3735 for ; Fri, 27 Sep 2002 18:13:03 +0200 (CEST) Date: Fri, 27 Sep 2002 18:13:15 +0200 Subject: Re: Re[2]: [workshop] rantelope day 003: the channel hierarchy Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Hans verschooten To: workshop@cornerhost.com In-Reply-To: Message-Id: <07138910-D234-11D6-BFA0-0030656FC5FA@net4all.be> X-Mailer: Apple Mail (2.546) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hydrogen.sabren.com id g8RGIC832749 Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: I looked at http://diveintopython.org/kgp_unicode.html. And there maybe a solution by setting sys.setdefaultencoding('iso-8859-1') But I need to set up my server at home so I can follow along with your coding. Cheers, Hans On Friday, September 27, 2002, at 04:30 PM, Michal Wallace wrote: > On Fri, 27 Sep 2002 11:17:38 +0200 Hans verschooten > wrote: > >> love what you're doing! Just one small point do you know that it's >> impossible to make posts with accented characters (יטא...) > > Hmmm. You're right. :/ Looks like it gets as far as the database and > RSS > feed, but the 4Xslt "minidom" parser can't cope with it. It's > definitely > some sort of unicode problem. I'm a complete newbie there. Anyone have > any ideas? > > Thanks Hans! > > Sincerely, > > Michal J Wallace > Sabren Enterprises, Inc. > ------------------------------------- > contact: michal@sabren.com > hosting: http://www.cornerhost.com/ > my site: http://www.sabren.net/ > -------------------------------------- > > _______________________________________________ > workshop mailing list > workshop@cornerhost.com > http://www.cornerhost.com/mailman/listinfo/workshop > From michal@sabren.com Sat Sep 28 00:17:22 2002 Received: from imf07bis.bellsouth.net (mail207.mail.bellsouth.net [205.152.58.147]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8S4HM809516; Sat, 28 Sep 2002 00:17:22 -0400 Received: from [192.168.1.102] ([67.33.117.68]) by imf07bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20020928041349.RIIV20456.imf07bis.bellsouth.net@[192.168.1.102]>; Sat, 28 Sep 2002 00:13:49 -0400 Date: Sat, 28 Sep 2002 00:05:40 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Subject: [workshop] rantelope, day 4: the ransacker search engine Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: rantelope: day 004 --------------------------- The Ransacker Search Engine --------------------------- Today I implemented a search engine that will eventually tie in to Rantelope. There's no new demo today, but links to the code are at: http://www.rantelope.com/ You also can now get the code via anonymous CVS: cvs -d:pserver:anonymous@cvs.sabren.com:/cvs/sixthdev co MODULE Where MODULE is python module name: (rantelope, ransacker, sixthday, storage, arlo, etc.) ==[ today's objective ]== Two years ago, I wrote a search engine called ransacker, and set it aside to work on other things. Now that rantelope is here, I thought I'd hook the two together. But the code was awful. Ransacker was always incredibly slow, so I did a bunch of tricks to optimize it. The result was an unreadable mess. So, today I rewrote it. ==[ implementation ]== So far, the code we've been looking at is mostly simple scripting of existing classes and objects. Since its mostly front-end work, I don't worry too much about testing. But today, I needed to build something useful, so I started by writing a test case. It looked something like this: ### import unittest from ransacker import MkIndex class MkIndexTest(unittest.TestCase): def setUp(self): self.idx = MkIndex() self.idx.add("onedog", "my dog has fleas") self.idx.add("cathat", "the cat in the hat") self.idx.add("twodog", "it's a dog eat dog world") def check_match(self): actual = self.idx.match('dog') assert ('onedog' in actual) and ('twodog' in actual), \ "match() doesn't find things: %s" % str(actual) assert actual == ('twodog', 'onedog'), \ "match() doesn't use relevance: %s" % str(actual) ### I have a program called 'sdunit' (sixthday unit tester) that runs tests like this for me, so all I had to do was save this into a new file called "spec/MkIndexTest.py" and type "sdunit". Of course, it gave me a big nasty error message because this code uses something called an MkIndex, and I hadn't written that yet. But that's the point. Writing this test let me think about what I wanted the MkIndex interface to look like, BEFORE I wrote it. - you add pages to the index - later, you can get a ranked listing back for a particular keyword. So if I indexed the three strings in the setUp() method, and then searched for "dog", I should get the names of the last and first strings. The "twodog" string has the word "dog" twice, so it should get ranked higher. I called it MkIndex because I knew I would be using MetaKit: http://www.equi4.com/metakit/python.html MetaKit is a small embedded database system. I had considered using it during the initial ransacker version, but never did. I had a hunch it would be a lot faster than the disk-based hash I used before. Adding a page looks something like this: - Build a frequency chart for each word in a page, which is actually quite easy if you have a dictionary data type: def wordFreqs(text): fd = {} for word in text.split(): if fd.has_key(word): fd[word] = fd[word] + 1 else: fd[word] = 1 return fd - For each unique word, add a [pagename, word, count] record to the database. So now we have an index. The match() routine just looks through the records for all pagenames that have that word, and returns them, sorted by the count. Turning that into code was just a matter of keeping the metakit documentation handy. Pretty soon, I had it working. It was definitely cleaner than the old code, but would it be practical for actual use? :: THE PROFILER :: Python has a built-in profiler module. A profiler is a tool that runs code and gives you statistics about which functions were called the most, and how long they took. It's a great tool for finding bottlnecks in code. I already had a profiling script for the old ransacker, so I figured it would make a good benchmarking tool. My test data was a single (long) page from http://www.robotwisdom.com/ . According to the profiler, the old but optimized ransacker took about a second to index it. Profiling code is easy: # idx = my index object import profile mycode = 'idx.add("pass1", txt)' profile.run(mycode) It prints a report with the total time and the function call statistics. Anyway, turns out new ransacker indexes the page in about 0.3 seconds. Still slow, but the old ugly code could hardly be called "optimized" anymore. :) But there were two problems: 1. So far, the metakit database was all in-memory, so we hadn't dealt with IO yet. 2. Since each record stored the word and the name of the page (which would be the URL in real life), and there's a record for each page/word pair, the database was bound to get HUGE. The first problem was easy. Just give metakit a filename. I did it, and the database went to disk, and it didn't hurt performance at all. To solve the second problem, I took a lesson from relational database design, and assigned each word and page a numeric key. So the index now looks like this: [wordID, pageID, count] The old ransacker worked that way, too, so I based my IdMap class on the old object. Both simply keep a disk-based hash that maps ID's to strings and vice versa. The code is simple, so I won't go into it here. It's in IdMap.py if you want to see it. Basically it works like this: im = IdMap() assert im["cat"] == 1, "should assign 1 to first word" assert im["dog"] == 2, "should assign 2 to second word, and so on" assert im["cat"] == 1, "should remember words already seen" assert im[1] == "cat", "the reverse works too" To store wordID and pageID, I just asked for wordIDs[word] or pageIDs[page] in the appropriate places. I expected this to add some overhead, but the speed stayed constant. Just for fun, I tried implementing IpMap in metakit. You can see my attempt in the "AllMkIndex.py" file. I abandoned it pretty quick though: it sent the indexing time up to 12 seconds. It probably could be sped up by caching results, but I didn't think it was worth pursuing. ==[ future directions ]== The new ransacker works, but it's hardly a full-featured search engine. One nice thing about it is that you can add pages to the index incrementally - like every time someone saves a new story. It doesn't let you update an old page in the index yet, but this is extremely simple: just delete the old records associated with that page, and add some new ones. Also, the index itself isn't indexed. Metakit basically looks at every record every time you search for a word. That's part of metakit's design: rather than indexing it just tries to be really really fast at brute force searching. Eventually, it might make more sense to store the table in MySQL, and add indexes to the wordID and pageID columns. Complex searches don't work. You can only search for one word at a time. Boolean searches like "cat AND dog", "cat OR dog", and "cat AND NOT dog" should be implemented by a higher level object, SearchEngine, that calls match() for all search terms and returns the proper subset of results. Finally, getting back to our content mangement system; if rantelope is to use ransacker, there has to be a way for rantelope to return a specific page. That means having a unique URL for each topic. That's not implemented yet, but it should be tomorrow, because we're going to build a comments system. :) --- PS: Tomorrow will be the last post out of me for about two weeks. As much as I'd like to work interrupted on rantelope and ransacker, I've got a hosting company to run... My plan is to have a 5-day period like this one every couple weeks. As always, comments, questions, and suggestions on the code are welcome, and feel free to forward this to a friend. :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From michal@sabren.com Sat Sep 28 16:26:32 2002 Received: from imf28bis.bellsouth.net (mail128.mail.bellsouth.net [205.152.58.88]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g8SKQW815346; Sat, 28 Sep 2002 16:26:32 -0400 Received: from [192.168.1.102] ([67.33.117.68]) by imf28bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20020928202031.UPHA3924.imf28bis.bellsouth.net@[192.168.1.102]>; Sat, 28 Sep 2002 16:20:31 -0400 Date: Sat, 28 Sep 2002 16:15:12 -0400 (Eastern Daylight Time) From: Michal Wallace To: workshop@cornerhost.com, workshop-lite@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Subject: [workshop] rantelope day 005: comment system and next steps Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: rantelope: day 005 ----------------------------- comment system and next steps ----------------------------- Today I added comments, rounding out the core feature set, and began thinking about how rantelope *the application* will eventually work. As usual, the code and demo are up at: http://www.rantelope.com/ You also can now get the code via anonymous CVS: cvs -d:pserver:anonymous@cvs.sabren.com:/cvs/sixthdev co MODULE Where MODULE is python module name: (rantelope, ransacker, sixthday, storage, arlo, etc.) ==[ today's objective ]== The last feature to add was a comment system. It's simple, so I'm just going to jump into the implementation here: ==[ implementation ]== I wanted to show a page with a story, list all the comments underneath it, and then show a form for people to add their own comments. Step by step, it went like this: 1. Create a Comment class and corresponding table: ## python ## class Comment(Strongbox): ID = attr(long) storyID = attr(long) name = attr(str) mail = attr(str) link = attr(str) note = attr(str) -- MySQL -- CREATE TABLE rnt_comment ( ID int not null auto_increment primary key, storyID int not null, name varchar(50), mail varchar(50), link varchar(255), note text ); 2. Define the relationship between Story and Comments: class Story(Strongbox): ... comments = linkset(Comment) ... dbmap = { ... Story.__attrs__["comments"]: (Comment, "storyID"), Comment: "rnt_comment"} 3. Add actions to RantelApp for showing stories and saving comments: def show_story(self): self.generic_show(Story, "sho_story") def save_comment(self): cmt = self.generic_save(Comment) self.redirect(action='show&what=story&ID=' + str(cmt.storyID)) 4. Create template called sho_story.zb. This is simple, but long. See: http://cvs.sabren.com/sixthdev/cvsweb.cgi/rantelope/sho_story.zb?rev=1.1 Pretty simple, huh? Of course, finding the comments page is kind of tricky, and it's not hooked up to the templates, and the search engine STILL isn't hooked up, and the whole application is just plain ugly. Which brings me to my next point... ==[ the future ]== Right now, rantelope is just a bundle of features. It leaves a lot of questions open. Is rantelope a tool users install themselves to manage their sites? Or should it be more like the blogger server, where people log in and publish their changes across the net? How should templates be shared between channels, stories, and comments? Do we need to implement a login feature, or can we delegate that to the webserver? If we do have users, how about multiple users, with different permissions based on the channel? When, exactly do channels get published? What about archives? How should the navigation be set up? What about email? Can you email a post to a blog? Can the blog notify subscribers of new stories? What about importing RSS? Or templates from other tools? There's no reason our various objects couldn't be stored directly in RSS files, bypassing MySQL completely. In other words, the future is wide open. A single program can't do everything, and that's where design comes in. I believe software design can and should be an interative process. A week ago, I wrote out which features would have to be there to build a content management system. Then I built each one in turn. The result seems like an ugly mess, but it more or less works. Actually, you might say that this week, we've created some technology for content management. Now we want to build an *application* of that technology, that people can actually use. So the next step is to figure out exactly how that application should interact with the world. I'll be working that out over the next two weeks. If you have suggestions, comments, feature requests, or questions, feel free to email me or post to the list. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.sabren.net/ -------------------------------------- From michal@sabren.com Tue Oct 15 15:47:56 2002 Received: from imf26bis.bellsouth.net (mail126.mail.bellsouth.net [205.152.58.86]) by hydrogen.sabren.com (8.11.6/8.11.6) with ESMTP id g9FJloT02649; Tue, 15 Oct 2002 15:47:56 -0400 Received: from [192.168.1.102] ([67.33.101.204]) by imf26bis.bellsouth.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20021015193740.BXVF8540.imf26bis.bellsouth.net@[192.168.1.102]>; Tue, 15 Oct 2002 15:37:40 -0400 Date: Tue, 15 Oct 2002 15:31:14 -0400 (Eastern Daylight Time) From: Michal Wallace To: innercircle@cornerhost.com, workshop@cornerhost.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Disposition: INLINE Organization: Sabren Enterprises, Inc X-Mailer: Mahogany 0.64.2 'Sparc', running under Windows 98 ( ) Subject: [workshop] what's your ideal blogging tool? Sender: workshop-admin@cornerhost.com Errors-To: workshop-admin@cornerhost.com X-BeenThere: workshop@cornerhost.com X-Mailman-Version: 2.0.8 Precedence: bulk Reply-To: workshop@cornerhost.com List-Unsubscribe: , List-Id: web app workshop - lessons + discussion List-Post: List-Help: List-Subscribe: , List-Archive: [crossposted to innercircle and web app workshop lists] Hey Everyone... I'd planned to work out a business spec for my rantelope blogging tool / content management system yesterday and post it to the webAppWorkshop list. For anyone who didn't follow the development articles ( http://www.rantelope.com/ ) I've built an incredibly ugly content management system, with all the basic features: post and publish, RSS, categories, templates, and a search engine (not yet hooked up). So I've got the basic technology, and now I want to create an application. The trouble is, there's a million ways I could build it. It could be like blogger (central server, remote publish), like movable type (cgi script, local publish), like livejournal (central server, local publish), or maybe something entirely different. More importantly, it could be a tool for managing a whole site, or a blog, or a link directory, or many sites, or... Whatever! I know what *I* want to use it for, but I'm not really writing it for me. I want to be able to offer a preinstalled blogging tool for people who sign up for hosting. So far I've not been able to work out a win/win deal with the owners of any of the famous blogging tools, which is why I finally decided to write my own. But my question is... What do people want? So I'm asking you all. :) - What features are most important to you in a blogging tool? - If you use one, why did you choose it over something else? - Would you want to use the same tool to manage the rest of