Using the new Monticello Importer from Instantiations


I just wanted to try the new Monticello Importer Beta that Instantiations just released yesterday and thought it wouldn’t hurt to describe the installation and use a little. Not that it’s hard, but not everybody might have seen monticello before.

Installation

Monticello’s .mcz files basically are zip files with a defined structure of ST source files and descriptions in them. The Monticello Importer therefor needs to unzip .mcz files to extract the ST code from them. So the first thing to do is install the zlib.dll. Instantiations recommends to use the one found at http://winimage.com/zLibDll/index.html. I downloaded the file zlib125dll.zip and extracted the 32 bit all from it and copied it to <vastroot>/bin.

You can find the Monticello Importer here. I won’t cover how to download the Monticello Importer from VASTGoodies, but if you need help with that, you can take a look at Marten’s screencast.

Getting Monticello Packages

Monticello has become the standard package format in the Squeak and Pharo stratosphere. They either use it to store local repositories on their machines or to access centralized, shared code repositories on the net. The first and most complete is squeak source. Due to some technical problems the community is working on new repositories, one of which is provided by vmware (GemStone), while there are other projects to provide the next generation, cross-platform Smalltalk code repositories.

Just like envy, Monticello allows to import code from local .mcz files or directly from a remote repository like the ones just mentioned. The current version of the Monticello Importer only supports importing from local files, so we need to download a .mcz from squeak source and store it locally.

I first tried with something quite simple: a simple CSV-file parser, which can be found at: http://www.squeaksource.com/CSV/. This page lists the latest versions of the mzz package and by clicking on the top-most entry you download the latest version to your local machine (this morning this was the file CSV-KasperOsterbye.8.mcz).

Importing the code

The file is here, the tool is installed, so let’s go. The current beta version doesn’t come with a GUI or menu item in the Transcript (which is not necessarily bad given the number of menu entries in the Transcript these days), so you start the importer by evaluating

MonticelloLoader loadMonticelloArchive

This will open a file explorer in which you select the .mcz file you just downloaded. The next step is to set the prerequisites for the newly to create Applications. This will probably be a bit hard to know right now, but you can change the app prerequisites any time later on, just as you do with your normal code. In this example which includes a bunch of test cases for the CSV importer code, the tool will suggest SUnit as a prereq, and that is a very good guess, so we just accept it for now. We’ll also be prompted for a version name for the newly imported Application(s). I chose “8 JTU 1″ for this example.

Now the Importer knows all it needs and starts importing. Here’s what it logs to the Transcript:

Info: 109 Made CSV(03.04.2012 05:57:50) from CSV Initial in CSV(03.04.2012 05:57:49).
Info: 58 New definition for CSVParser(03.04.2012 05:57:49) in CSV(03.04.2012 05:57:49).
testComaIsDefaultDelimiter
 | csv csvParser |
 csv := 'Luke,Skywalker,"Young Jedi"'.

 csvParser := CSVParser onString: csv.

 self
 assert:
compiler error "invalid character sequence"
--> {'Luke'. 'Skywalker'. 'Young Jedi'} asOrderedCollection
 equals: csvParser rows first.
testWithSemiColonDelimiter
 | csv csvParser |
 csv := 'Luke;Skywalker;"Young Jedi"'.

 csvParser := CSVParser onString: csv.
 csvParser useDelimiter: $;.

 self
 assert:
compiler error "invalid character sequence"
--> {'Luke'. 'Skywalker'. 'Young Jedi'} asOrderedCollection
 equals: csvParser rows first.
Info: 42 CSV class overriding #loaded from SubApplication class.
Info: 42 CSV class overriding #removing from SubApplication class.
Info: 42 CSV class overriding #failedRemove from SubApplication class.
asJSONWithHeader: aCollectionOfString
 |rows|
 rows := self rows copyWithoutIndex: 1.
^ String streamContents: [:aStream|
 aStream nextPutAll: '['; cr.

 rows do: [:row |
 aStream nextPutAll: ' {'.
 aCollectionOfString do: [:property|
 aStream nextPutAll: (' {1}: "{2}"' format:
compiler error "invalid character sequence"
--> {property asString. row removeAt:1})
 ]
 separatedBy: [aStream nextPutAll: ','; cr].
 aStream nextPutAll: ' }'.
 ]
 separatedBy: [aStream nextPutAll: ','; cr].

 aStream nextPutAll: ']'.
 ].
Info: 42 CSVParser overriding #initialize from Object.

After the import finished, there will be a new Application created in your image (and library). In our case it is an Application called CSVApp 8 JTU 1 with a SubApplication called CSV 8 JTU 1. The Application itself is empty, all the code has been imported into the SubApplication. In this example that’s only two classes: CSVParser and CSVParserTest.

That’s basically it. The importer compiles all methods it can compile and for those it can’t , it does something different from what you’re used to.

Normally, when code cannot be loaded from into the image, you get a scratch edition of the Application and methods that cannot be compiled are missing from the classes.

The Monticello Importer, however, imports the code, but comments it out and adds a halt: to the resulting method. Let’s take a look at this one from CSVParserTest, for example:

testComaIsDefaultDelimiter
"The following source failed to compile (all double quotes have been doubled):
 | csv csvParser |
 csv := 'Luke,Skywalker,""Young Jedi""'.
 
 csvParser := CSVParser onString: csv.
 
 self 
 assert: {'Luke'. 'Skywalker'. 'Young Jedi'} asOrderedCollection 
 equals: csvParser rows first.
"
^ self halt: 'The source for this method failed to compile'

You see, Squeak has a special notation to create arrays in curly braces that is unsupported in VA Smalltalk and therefor the code cannot be compiled. The Monticello Importer makes sure we don’t lose the code, but it doesn’t attempt to port it.

You also see it doesn’t correct orthographic errors ;-).

So what’s next?

The Monticello Importer helps you get the code transported into VA Smalltalk and provides you with some help in starting the port, but porting is a manual task. First of all, after importing, it sure is a good idea to look at all senders of halt:. You should now fix all the commented pieces of code. If you’re lucky, you’re done after that. Very often, you’re not. The imported code may send methods that are not in your image, either because they’re squeak/pharo specific or because they use code which is currently not loaded in your image. This may require to import even more code (like a squeak specific HTTP Socket implementation) or change the imported code to fit well into the VAST ecosystems. This is a process of trial and error and may or may not be a easy.

My advice would be to first concentrate on the test cases shipped with the .mcz. Testing is well adapted in the Squeak/Pharo world and the test cases tell you a lot about how good your port is. Then start adding your own tests for VAST specific changes you make.

Once you’ve ported code, please feel free to share: VASTGoodies is a friendly place and chances are other VAST users can make good use of the goodie. And also do the original authors of the goodie a favor and let them know about problems you found, things that made porting hard (there are quite a few open source projects out there that care about cross-Smalltalk portability) and let them know you’re using their code. They’ll be happy to hear about it. If you fixed a bug, contribute it back.

And what is missing?

All of this, of course, is not the whole story: The Squeak and Pharo crowds have long built tools on top of Monticello to handle dependencies between packages and to support loading of complex code bases. Gofer and Metacello are the most important ones to mention. We’ll have to see whether somebody will come up with tools to support loading a ConfigurationOfXXX into VA Smalltalk.

And then, there is another step on the journey. The VA ST developers may now be able to harvest open source code and port it to VAST much more easily than before, but some might really like to contribute to the open source world outside of the VA ST ecosystem (this has been a problem for all ST vendors). There might be extensions or bug fixes to existing projects or even completely new projects that currently only live on VASTGoodies, where no other ST dialect’s users can load it from. So what we’ll need is some way to contribute back. For now, this will require the use of Pharo or Squeak or GemStone and manually reimplement (or copy&paste) your fixes there. Hopefully there will be a solution that will allow for some easier cross-dialect code exchange sooner rather than later.

The VAST continent is a young one in a geologic perspective on open source, but it is moving out of the shadows. There are quite a few nice projects on VASTGoodies that might help not only VA ST users in their daily lives, and I hope we’ll find ways to contribute to the greater ST community more easily. As sad as it sounds, it is important to note that other dialects have or have had the same problems until just recently, StORE, for example, is as incompatible with Monticello as is Envy.

About these ads