Build automation is social engineering

James comes up with an interesting observation:

It’s one thing to come up with a few simple tools to automate a build […] The harder problem is fitting the technical siolution into the process that your project has evolved.

I’ve been in that boat a few times (You can have a look at a talk of mine from last year’s VA Smalltalk forum here (PDF) (Video)).
While it seems easy to define a standard way of managing code, building and deploying artefacts and such, it can be extremely hard to use that way of doing things work in a project team that has

  • existed for 10 or more years
  • lost the people who once managed all that stuff
  • always done it the way they do it (often without understanding why)
  • never enough time to change such complicated and unimportant stuff (because improvements in the process do not immediately improve the product)

So I’ve given up on the idea of giving teams a toolset for managing their code and do automatic builds. This will surely fail, even if the tool involves automatic build servers like Hudson (which is really easy to use for Smalltalk builds). Because the hardest part is not to make Hudson or any other rool run. The hardest part is to change a team’s habits in order to deliver a codebase that can be handled by the tools.

My approach is to work with a team for a while and tweak my ideas and tools individually for that team or develop new concepts and tools with the team rather than trying to tweak the team too much. It’s good to identify problems and convince people to change their work style, but you can only go a certain mileage into that direction without risking to mess up the project completely or become the esoteric clown that nobody listens to any more.

It is much better, btw.,  to first see what problems the team has identified and try to solve these first. Many times, these have nothing to do with Build Automation at first sight, but with lack of communication or knowledge.

Build automation is social engineering – at least in existing project teams.

Commercial Break: If you are looking for a social engineer with knowledge in Build Automation in VA Smalltalk, perform these two easy steps:

  1. look at one of the above-mentioned links to my talk
  2. get in touch with us if you like what you see

 

GLORP on VAST 8.03

VA Smalltalk ships with GLORP (a relational database mapping framework with lots of ideas and concepts from TOPLInk)  since the latest VAST version 8.02. Since the port of GLORP shipped with VAST is based on an older version of GLORP and also is not really well integrated with VAST, its use in production code is not really recommended. Instantiations’ recommendation here is to use GLORP as it is right now more to learn GLORP and start porting efforts.

Things will hopefully be much better with VAST 8.03 or the release after that. And John will be working on this with one of the (if not THE) GLORP porting experts:

Niall Ross and I will be doing some work on GLORP at Camp Smalltalk this weekend in Barcelona — the platform-specific code (such as the code you referenced) is high on the list of things to address.

VA Smalltalk and open source databases

VA Smalltalk has its traditional background in large corporations where most of the data is stored in the 800 pound gorillas DB2 and Oracle. The support for these databases is very reliable and stable, and Instantiations keeps their support code up to date with the supported products.

But open source has reached the enterprise in many areas, including databases. The web designers brought in MySQL with their PHP sites and Postgres is not particularly unsuited for many things that typically were driven by DB2 in the past.

So the time has come to support these databases as well in VA Smalltalk. Of course you can always use an ODBC driver and get VA Smalltalk to talk to these DBs, but ODBC hast ist issues.

Marten has put quite some work into PostgreSQL and SQLite support for VA Smalltalk. In his latest Blog post he gives a short introduction to how far he got, what problems he found and where he could need a little help as well as a short introduction on installing and using it.

You can always download his stuff from VastGoodies.com.

I think we should have stable support for at least PostgreSQL, MySQL and SQLite in VA Smalltalk in order to give newbies a chance to get to speed with VA Smalltalk very quickly, for example if they want to port their Rails application or a PHP based web site over to Seaside. SQLite is especially interesting for (small) businesses who want to ship packaged end-user applications (home banking, shopping list management, accounting, media management etc.). Not all and everything today is written against a NoSQL database…

An easy migration path is especially important for users who came to Smalltalk via Squeak or Pharo, but need the ability to produce GUIs with native widgets. They should not be forced to switch databases as well, just because they want to use VA Smalltalk.

So Marten’s work is really essential here, and I hope we can improve on his foundation.

The future of VA Smalltalk

Now that we’ve learned that Instantiations is turning into a pure Smalltalk shop, it’s probably time to take a second look at what Mike Taylor (the old and new CEO of Instantiations) said in his talk at the VA Smalltalk Forum Europe 2010, back in June.

I’ll let the following slides of his talk speak for themselves:

You can download this and all other talks of that event from our webiste. If you’re interested in more details, watch the video of Mike’s talk.

VFE2010: all presentations now available as videos

All presentations given at this year’s VA Smalltalk Forum Europe are now available for download as pdf-documents and as mp4-videos (great quality, but also big file sizes).

You can download all the slides and videos on our download page.
I’ve also uploaded my presentation to Slideshare.
And there is of course Adriaan’s VFE2010 photo gallery.

So if you missed the event, you can now get all the info you’ve missed, but you still cannot experience how great it is to meet other Smalltalkers face to face.

A special Thank You goes to Marten Feldtman, who put the videos together and to Adriaan van Os for his photos.

VFE2010: Video of my talk is online

Marten Feldtmann is working his way through a hole day’s material of recordings of our VA Smalltalk Forum 2010. He’s just finished editing the video of my talk that and put it online. You can view or download it from his blog.

I’m surprised by the high sound quality of the vidoes and I’d say Marten did a great job of recording and editing the sessions. I want to repeat my Thank you for that!

PragDave on Software Archeology and The Advantages of Self-Containment

I just finished listening to Episode 148: Software Archaeology with Dave Thomas (Pragmatic Dave) of Software Engineering Radio.

It’s a really interesting interview and Dave has a lot to say about our Software Engineering Jobs of today:

  • Most of the time we read code rather than write it
  • Most of the code we have to read is in less than perfect shape
  • Most of the programmers aren’t educated in how to read code
  • Far too often people change code they don’t really understand
  • Tests are a great way of proving your hypotheses about what existing code does
  • Code rots, and you have to be prepared or it
  • Only documentation tends to rot faster than code
  • Most documentation that gets written is mostly useless, because people tend to document the obvious rather than the reasoning behind their decisions
  • Reading code is not only good in software maintenance but also is an educating exercise
  • You can write bad code in any language, but also: a well-written program can be easy to understand in any language

There’s not much space for disagreement here, since you learn these lessons pretty fast when you maintain software. Also the idea of having to “get into people’s heads” about their style of programming, thinking and problem solving is absolutely right. To me, it’s often good to know who wrote a method, especially in projects where I know the initial programmers. You sometimes really know why something was done in a certain way if you know who wrote the code.

So it may sound funny, but sometimes you find your way to a bug a lot faster by knowing whose code it’s in. Every programmer leaves some DNA in their code, and since software maintenance has a lot in common with detective stories, it’s always good to have a profile of your “Mister X” in mind 😉

Some of the things Dave talks about sound weird or ridiculously obvious at first:

  • Print out an artefact in 2pt font size to learn about its structure
  • You need to make sure you’re digging in the right version of the code
  • You need to make sure you have all the necessary code
  • Use a local code repository
  • IDE’s and code generation can be dangerous
  • Tools like Emacs and grep can be of great help

But once you think about them, you’ll find them much more useful advice than they sound in the first place.

When trying to translate some of these to Smalltalk, however, I guess we can put some of these aside. There’s no point in using grep on Smalltalk code if you work in an image based environment. There are much better tools like References, Senders/Implementers and such. And there’s not much use in looking at a body of code isolated from the Smalltalk class library. In the light of these arguments, the use of a good source code control system like Envy becomes more important. Since I never tried the tiny font-size thing, I can not really judge it. But I can imagine that in a purely file based world, even this kind of thing can help (and if not, it’s been worth a try).

This, combined with the fact that software archeology is not necessarily a nondestructive task (it’s not about conserving code, but about understanding it), version control gets more important. As I mentioned in my talk at the VA Smalltalk Forum two weeks ago, Refactoring is a great way of finding your way into existing code. Sometimes just changing the name of a temporary variable or a parameter or extracting parts of e method and giving it an intention revealing name makes understanding code a whole lot easier.

Dave also mentions the importance of getting a hold on all artefacts that are needed to make a piece of code run, like database creation scripts and stuff. Since I am a strong believer in “get all your stuff into your code repository” and since my favourite language Smalltalk makes this quite easy by having the compiler available to create classes out of scripts (again, see my talk for an example), I think we have a very good environement to our disposal to make the lives of later software archeologists easier.

Since Software Maintenance is one of the most important and least appreciated tasks in Software Engineering, I guess we owe it to ourselves and to the developers who need to get along with our code to use techniques and tools that make archeology easier. We need to improve our coding style, documentation habits and we need to keep in mind our code is not only there to be executed by machines, but mostly to be understood and reworked by humans.

And I guess there aren’t many environments that make reading (into) code as easy as Smalltalk. The Smalltalk Debugger as a means of watching the innards of a program on the job and even modifying a program in the middle of a task, together with a rich set of inspection tools and well-integrated testing facilities like sUnit, combined with a whole set of Code Browsers that are written around the idea that “FINDING EXISTING CODE” is the main task in programming, not only make writing code easier in the first place, but also help later developer generations to understand our code.

In case of Smalltalk, this is not a theoretical assumption, it is a proven fact that the language allows for long product lifecycles. There are many systems around that are running for more than one and a half decade, and continue to be integrated into modern architectures like Web Services or to be transformed into Web based applications while also adding new features all the time. This would presumable be a lot harder in less maintenance-friendly environments.

As developers, it’s our responsibility to lay the grounds for later generations, and we have to make decisions that will have consequences even when they’re pushing us through the park of our elderly’s home on a wheelchair every wednesday afternoon. So using an environment that’s helpful in reading code and checking hypotheses is what we should do. Smalltalk is an environment that qualifies in this area and should seriously considered as the platform of choice.

Many people out there think that Image-based systems are a bad thing, since they feel they’re losing control over their system’s structure, but it’s quite the opposite: A Smalltalk Image is the whole world of the objects in a system and therefor gives you much more abilities to explore and search than a list of dead files. In the end, in an object world, what kind of structure does a file give you? Your running program consists of classes and instances, not of files.

Taking Dave’s Analogy a step further, there is a strong advantage in Smalltalk, because learning about existing code is not really archeology, but much more like time travel: you just jump into the bazaar of a running system, talk to the merchants and visit the temple on the hill instead of just digging some old stones out of the sands. So while software archeologists try to imagine how a system might work or have worked, we Smalltalk developers have the luxury of just starting the system up and live in there for a day or two.

VFE 2010: Attendee fedback

At last week’s VA Smalltalk Forum Europe 2010, we asked ourattendees for feedback. Based on the forms that were returned, it seems we did quite well:

  • 77% of the attendees saw their expectations met, while 23% partially got what they were looking for
  • 64% found the organizations of the event very good, 33% thought it was good
  • 94% said they’d recommend the VA Smalltalk Forum Europe to their colleagues and Smalltalker friends

Since we’ve had an extremely packed and interesting agenda this year, the top most point of critique was that there was too little time for networking in the coffee breaks and at lunch time. We’ve gotten the message and will reduce our agenda for next year by at least one session / hour and plan for longer breaks. The fact that two sessions were held remotely also was a point that was criticized, because the feedback loop is too long to ask detailed questions. And, it would have been nice to discuss more details at a coffee table.

We also asked for individual feedback on all our sessions, and all I can say is that it seems we pretty much hit the nerve of our attendees topic-wise as well as with respect to the quality of the talks. The top score went to Lukas’ Seaside talk, which is no surprise to me at all.

By the way, you can download all presentations [link was updated] from various places. And, even better, Marten has recorded all sessions and is currently working on editing them. A few have already been put online.

People liked the food and the location, which was very practically situated directly at the main train station in Stuttgart. So coming to and leaving from the event was easy. I guess the location was chosen not too badly, since we had attendees from Germany, Switzerland, The Netherlands, Liechtenstein, Belgium and the UK.

I guess our feedback shows that you really missed something if you were hesitant to come this year.  But I’m quite sure there’ll be another chance. There are no specific plans for a next event yet, but I guess we’re going to organize another VA Smalltalk Forum Europe next year.

VFE 2010: Slides of My Talk: Get (Your Project) Back to Shape

If you’ve missed the VA Smalltalk Forum Europe 2010 in Stuttgart last week, there is at least a chance to take a look at the presentation slides. While we’re still working on an “official” central version location for all the material, I’ve put up some files to our company website, where you can look at them.

My talk was mainly about improving your working style in legacy Smalltalk Projects. Truth be told, there are much more Legacy Smalltalk Projects out there than greenfield projects, even though the Seaside Frameworkl has attracted quite a few Developers into the Smalltalk community.

Legacy here is not a negative term. Many Smalltalk projects are legacy projects, because they’ve been running for more than a decade, fulfilling their business needs and making their users happy. Since there has been a lot of pressure in big corporations towards a “unified platform” (the name may vary in your company), the projects had to survive under really unpleasant circumstances:

  • fluctuation of key developers to newer projects
  • shrinking budgets
  • an aura of being unwanted, old technology
  • neverending wishes for new features

In some organizations, this has lead to a very “pragmatic” (this is not the “pragmatic” of the pragmatic programmers movement!) approach to implementing new features and bug fixes:

  • touch as little code as necessary
  • forget about how it should be done
  • ignore all the new techniques like unit testing, refactoring, build automation etc.

A few years ago, this sounded like a good idea, because people would never have to touch the code again, when the system would be switched off a few months later.

Things went different (as they usually do):

  • The new projects failed completely or in part, so that they were cancelled or reduced in scope
  • The new projects were so expensive that it was decided to couple the old and new systems and keeping the core logic in the Smalltalk system
  • The new projects didn’t really stay within their timeline (surpise!) and so the old one is still the only thing the users have. The result is feature creep in both projects, which may possibly lead to one of the above-mentioned consquences in the future. The difference for the end users is very relevant: they never see the new system anyways, but they get the new features in the old one.

At the end of the day, all of this means the Smalltalk projects in this situation are faced with a few problems:

  • Bad code stuffed into a once good and clean system (Smalltalk Projects were generally well-staffed and budgeted back in the early to mid nineties)
  • Techniques that are common sense today (and, ironically, were invented in the Smalltalk world, like Refactoring and unit testing or automated builds) never got adopted in these projects
  • The once highly-skilled team has lost a lot of talent (Most organizations have shot themselves into their knees by forcing good people out of the projects and scaring everybody away from them by officially calling them a dead-end)
  • Lots of features need to be implemented and the all the ugly crimes of Software Maintenance that have been committed “for the good” need to be cleaned up now. The reason for this is that the systems have returned to an “open end of life” phase, meaning there is no question that these projects will continue to run for another 5 or ten years or “forever”. This, of course, means you need to handle existing code with a lot more respect than people have done for quite a while

My talk is a short summary of findings around all this. I’ve been helping a number of projects to cope with exactly this situation over the last few years by teaching or coaching and by introducing technigques and tools to make the developers’ lives easier. I demoed a few of the little helpers for coping with typical problems like artefacts that are not under version control and tried to motivate people to use the refactoring browser as a tool for reading into existing code (I’d say Smalltalk is one of the best environments for software archeology).

Another area in which projects have serious problems is configuration management with Envy/Developer and the build process. This is not so much a technical problem but more a problem of fear and misunderstanding as well as a question of complete lack of tools and automation in that area. I’ve shown some of the stuff I’ve implemented for customers to overcome these problems. We’ve implemented a complete set of tools and helpers for code management in Streams and automating builds for a Continuous Integration Environment.

Once the video of my talk is available, you’ll probably get an impression of how much I enjoy working with such project teams. There’s a lot you can achieve with little steps and it’s really motivating to see project teams gaining self confidence back that was lost in-between a lot of frustration and sadness about the ignorance of their own management. And, even more important, it’s a lot of fun pairing with a programmer who discovers the power of the Refactoring Browser and Unit Tests.

So if you like my talk and think your project could use some of the tools and techniques to get back to speed again and if you need someone to scratch the rust from your code and make your project shine again, don’t be shy and contact me.

VFE 2010: The first videos are online

Marten Feldtmann was kind enough to not only attend our VA Smalltalk Forum Europe 2010 in Stuttgart last week, he also recorded all the sessions and is now making them available over time (it can be lots of work to get a video ready for release):

I’ve finished the preparation of the first three videos of this event. I think, that Instantiations will publish these videos also.

The first videos are:

* VASmalltalk Business Update – Session 01
* Smalltalk State – Session 02
* WebServices – Example – Session 07

So head over to Marten’s Blog to have a look at the vidoes.

So, a big THANK YOU goes to Marten!

Meanwhile, we are working on getting all presentations slides, remote session recordings, pictures and Marten’s vidoes into one place on the web for you to download them. Stay tuned.