Neue Diskussionsgruppe zu Smalltalk in Deutsch

Nachdem Xing aus mir ziemlich unverständlichen Gründen die Gruppen abgeschafft hta, hat Helge Nowak eine neue Gruppe auf LinkedIn eingerichtet. Hier seine Einladungsmail:

Hallo Smalltalker!

Die weltweite deutschsprachige Smalltalk-Anwendergemeinde hat einen neuen Treffpunkt: die GSUG-Gruppe auf LinkedIn: https://www.linkedin.com/groups/12754122/

Hier wollen wir uns zu allen Themen rund um Smalltalk austauschen: Technologien, Produkte und Projekte vorstellen und diskutieren, Stellenangebote und -gesuche bekanntmachen, und was uns sonst noch zu unserer Lieblingsumgebung bewegt. Die Gruppe ist dialektneutral und offen für Erweiterungen im Geiste von Smalltalk, wie Self, Strongtalk oder Newspeak. Wir sind schon gespannt auf die nächsten innovativen Ideen!

Wir freuen uns auf Euch! Anmelden und Weitersagen 😉

Smalltalk ist anders – und das ist gut so!

Helge Nowak

TOTP in Smalltalk – a little experiment

Being a Smalltalk developer gives you lots of joy; Smalltalk is such a nice programming environment, The degree of interactiveness of the toolset and the expressiveness of Smalltalk code are legendary.

But being a Smalltalk developer also put some burden onto you.

The community is quite small, the dialects of Smalltalk still have no common sense of what an exchange format for source code between IDEs should look like and differences between dialects are big enough to make code exchange much harde than it should be.

This. combined, leads to some interesting problem: chances are that the selection of ready-made code for your problem is small.

Example: There is no Smalltalk dialect that can parse Mime-mails correctly. Seems nobody ever needed this. There are implementations that can get you quite far, but they still fail in edge cases. In langiages like Python or Javasvcript, there is a ton of modules and frameworks for all kinds of problems. Not all of them are perfect or even close, but there is a wide selection of alternatives to try.

In Smalltalk you often end up reading RFCs and feeling like the first to implement stuff that is just a PIP away in Python. Or an npm install. There are lots of problems coming with that as well, so the grass is not always greener on the Python side. You still have to do research and find the right module for you. But they are there, you can try them. In Smalltalk, there might be a module, but often you are out of luck or need to port one from another dialect.

One of these problems that are unsolved in Smalltalk – believe it or not – is a publicly available implementation of TOTP (Time-based One Time Passwords). There’s a little bit of encryption and bit pushing involved and there is nothing really hard going on. But you still have to pick up the pieces and combine them to a working solution.

Marten, Thomas and I had a little chat about this specific issue and decided to do a little online session and take the time to implement something.

The plan was to quickly go through the spec and nail together a first prototype. The interesting thing here is: each of us works primarily on another Smalltalk dialect 😉

After an evening hacking together and playing with it, we quickly got to the point where we could register a secret in freeOTP and verify its passcode in Smalltalk (we used Gemstone, because Marten just typed as we talked).

After a good two hours we ended the discussion with a what’s missing and how we’d solve the problem of generating the QR-Code for registering a secret with freeOTP, Authy or Google Authenticator.

Marten was the first to finish the code up and publish an article about his solution on his blog – including source code. So you can now implement your own solution in your Smalltalk dialect using his code snippets. We didn’t go into detail about all the bells and whistles an application would need to make a 2FA login process workable, like for example

  • How would a user register for 2FA or disable it
  • What would a mechanism for cases like “I lost my phone and forgot to create a new secret”, “I don’t have my phone available at the moment, but I need to login anyways” look like
  • We haven’t given too much thought on Timezones, so there may be more to 2FA than we currently implemented

I need to adapt the code to my VAST Platform project and am planning to publish it on github. I just need to find the time to do so. Maybe someone will find use for it and probably suggest improvements (like the Timezone problem).

One issue will remain for this as well as most other Smalltalk projects, though: how can we make this one codebase in which bugfixes and imrovements can be implemented in one place and make them available on all target dialects? Still one of the biggest hurdles for wider acceptance of Smalltalk as an ecosystem spanning multiple of all dialects.

Glorp and embedded objects – how null is nil and why not?

Hi, it’s been a while. I am still trying to find the time and energy to write posts more regularly. One of the things I had planned was to write about things I learned, especially when I learn something new about (or even solve a problem with) Glorp, my favorite Object-Relational mapper.

It’s almost New Year’s Eve, so this post is proof that it’s never too late to work on your new year’s resolutions, even if they were made a few years ago. So, to make things complete on this side thread: Happy New Year 2022 everybody! Let’s hope we’re finally getting ahead of the pandemic wave and get back to a new normal life.

But now for the problem I am proud to have solved on my own 😉

I have this Smalltalk class named DateRange in my Application. It only has two instance variables: #startDate and #endDate. But it bundles a lot of logic having to do with whether a DateRange overlaps with another one, if contains a Date, starts or ends before or after a certain Date or another DateRange, can be used to sort objects by the DateRange it covers, etc. You get the picture.

The business logic of this DateRange is so helpful in my domain (Accounting) and occurs so often that I wrote this Class and was looking for ways to store DateRange instances in our relational database as two columns in the table of the containing business objects. So a business year has a start and end date, the duration of usage of some good you use in your business has a start and an end date, a Tax Report has a startDate and and endDate. But it makes absolutely no sense to store all DateRanges in their own DateRange table in the database and use 1:n relations to load/store these DateRange objects. It is much better to store the #startDate and #endDate as columns in the BusinessYear table or the TaxReport table, but still have Glorp retrieve the combination of these two attributes as an instance of DateRange.

The good news: Glorp can do that. It Provides an EmbeddedValueToOneMapping. This Mapping is super handy and clever and it even allows you to define the concrete column names in a table for your startDate and endDate in this concrete table, so that you can even store multiple DateRanges in a row if a business object has more than one DateRanges (like, say, a period of availability and a period of vailidity). Thus you can store multiple DateRanges in one row, each having separate column names, but still mapped to the same Class.

The Mapping in the Descriptor looks like this:

(aDescriptor newMapping: EmbeddedValueOneToOneMapping)
    attributeName: #availabiltyPeriod;
    fieldTranslation: (
        (Join new)
            addSource: (table fieldNamed: 'avail_start_date')
                target: ((self tableNamed: 'DATERANGE_EMBEDDED') fieldNamed: 'startDate');
            addSource: (table fieldNamed: 'avail_end_dat')
                target: ((self tableNamed: 'DATERANGE_EMBEDDED') fieldNamed: 'endDate');
            yourself).

For this to work, you also need a special mapping that knows how to convert two dates to a DateRange and back. It is called an embedded table mapping and looks like this:

tableForDATERANGE_EMBEDDED: aTable
  aTable createFieldNamed: 'startDate' type: platform date.

  aTable createFieldNamed: 'endDate' type: platform date.

This is extremely powerful and works great. You can store and load objects with DateRanges in instance variables and get DateRange instances back from the database.

…until the DateRange is nil. And by “is nil” I mean a DateRange that is not present in an object. In our example above, we have a business object whose #availabilityPeriod is nil instead of an instance of DateRange.

Glorp is clever enough to save this DateRange as two NULL values in the avail_start_date and avail_end_date columns. So far so good.

If you are as old as I am, you know that movies back in the 70ies and 80ies had time. By this I mean you could have sequences in movies where nothing happened for a minute or even longer. You just watched some object flying away from or towards you in space, or the camera showed you a long road through a desert and after 20 seconds or so you could see a car breaking out of the horizon and you watched it while it drove to the current camera position for 50 seconds, diving down some hills, diving out of our view and come back to view a few times while growing bigger with each hill.

They don’t make movies like that any more, but I like the idea that someone is still reading and wants to learn more about my Glorp problem, so take this as the long winding road leading up to the climax of my post 😉

When Glorp reads back an embedded object, it will always instantiate on Object of the mapped class, in our case the DateRange. Only after instantiating the DateRange it will populate its instance variables.

So the end result of storing the content of the instance variable availabilityPeriod that is nil and reading it back will be a DateRange with a startDate of nil and an endDate of nil. Which, obviously, is not the same as nil. This may or may not cause all kinds of problems, especially if your business code does #isNil checks on your availabilityPeriod variable. Before you save an object to the database, availabilityPeriod is nil. As soon as you refresh the object or load it back from the database it will not be nil but a DateRange containing the startDate nil and the endDate nil.

Ouch.

Let’s watch the evil guy enter his car and drive back to wherever he came from for a while. In former times, we’d be watching this and enjoy this quiet moment and maybe feel a host of mixed emotions that the director of the movie wanted us to feel. These days, viewers aren’t trained to enjoy such scenes any more. Maybe you take this moment to think about the consequences of Glorp’s strange behaviour and ways to possibly solve this problem instead.

I am far from being perfect as a developer, but here are options that I came up with:

  1. I could try to make DateRange behave like nil (objects are all about polymorphism, right?) when all its instance variables are nil
  2. Wait: there must be an option to tell the mapping to not instantiate anything if all columns are NULL and return nil instead

While the first option sounds not too bad at first, let me tell you it is actually a really bad idea. Using DateRange will turn into a game of Minesweeper without the hints on uncovered fields. Almost every method in a DateRange will have to start with the philosophical problem of whether this DateRange is possibly not actually a real DateRange but something nillish. Or somethingd like that. Let’s not go into the details here, you’re still reading, so I assume you accept I tried and proved the idea to be bad.

Of course I started my journey with option 2. of the above list. I was looking for ways to tell Glorp to not instantiate a DateRange if both startDate and endDate are nil (or better: their respective columns in the DB are NULL). But I couldn’t find any attribute in EmbeddedValueToOneMapping that is capable of doing it. (If you don’t believe that I looked and also searched for help on this, follow this link to see I did). And I gave up.

Until I encountered this problem a second time, and this time I wasn’t ready to give up.

And it turns out there is a way, but it doesn’t work by configuring the mapping.

I guess you are not interested in whether the bad guy’s car is red or what brand it is, so I’ll skip this section and come right to the solution. I’ll even save you from reading what I did to find it. It’s an interesting story per se, but you’re not paying me for anecdotes, are you?

During the building phase of an Object, each newly created and populated instance gets send the message #glorpPostFetchValidate.

If this method answers anything else than self (which, as you know, is the default return value of every Smalltalk method), Glorp will populate the mapped instance variable with nil. So I implemented this instance method in DateRange:

glorpPostFetchValidate: aSession
	"This allows us to do post-read notification of the objects. Note that if this method explicitly returns a false, then we will treat that as meaning that the object is invalid and should not be read. Yes, this is kind of a hack."
	
	(self startDate  isNil and: [self endDate isNil]) ifTrue: [^false ]

So if I now read objects with DateRange mappings, the objects will contain a real nil as value in their instance variables.

myDbSession read: Product where: [:id| id = 5483].

Now has an availabilityPeriod of nil. All existing code still works

You’ve reached the end of the story. Thanks for staying and reading. I know there’s little plot for a long post. Like in a cult movie from the past.

But wait, there is something strange I would like to mention.

Remember the movie Coming To America starring Eddie Murphy as Prince Akeem Joffer? If you stood up when the end credits start scrolling over the screen, you missed the best part. These jokes after the credits still occur in movies today, so even if you are young enough not to know the movie, you probably know the parts of Ice Age when Scrat fights the facts of life after the credits. If you don’t, never mind.
There is a part of the EmbeddedValueToOneMappings that leaves me stunned.

If you query the database with a query like this:

session read: Product where: [:p| p availabilityPeriod = nil].

Glorp still knows how to convert this to correct SQL, because it will create a statement like this:

SELECT t1.id, ..., t1.AVAIL_START_DATE, t1.AVAIL_END_DATE
 FROM PRODUCT t1
 WHERE ((t1.AVAIL_START_DATE IS NULL) AND (t1.AVAIL_END_DATE IS NULL))

So Glorp is pretty much aware of what nillishness of an embedded object means. It just doesn’t use this knowledge at read/build time. Or I am still missing something – maybe someone can enlighten me.

I wonder if this is more a bug or shortcoming in Glorp or if I am just expecting too much…

Instantiations to present their plans with VA Smalltalk

Instantiations is going to present their history, present and plans with and on the basis of their VA Smalltalk platform for the future on the next online meetup of the Buenos Aires Smalltalk Meetup on May 10th, 2021.

This is going to be a free event on zoom.us and here is what you can learn at this 2 hour presentation:

Instantiations has attracted some attention with its new branding, new website, and the release of VAST Platform 2021. Join members of their staff as they discuss the evolution that’s been occurring over the past 30 years and how it made reaching these milestones possible.
Instantiations will discuss the direction their technology company is headed, provide insights about how they’ll get there, and demo the technical breakthroughs that have been recently released and are upcoming.

You can register for this presentation fro free and see what Instantiations is up to:

https://www.meetup.com/de-DE/BA-Smalltalk/events/278106330

Look what I’ve got ;-)

I am really eager to explore the new Native OsProcess Framework in VA Smalltalk 2021 (aka 10.0.0). There is a use case in Kontolino! for it that will greatly profit from CHaining external processes and having more control over their suceess/results.
And the new Futures are waiting to be researched as well….
So far I can only tell that installation worked like a charm, my old images work as expected, which is a good start.

Election Reporting System written in Smalltalk (Gemstone/S)

Over on his Blog, Marten showcases a real world system written in Gemstone/S Smalltalk for collecting and predicting as well as reporting election results on German TV. The Backend system runs in Gemstone/S and has been in use for that purpose for several elections in German federal states for a few years already. So if you watch the special programs on German ARD or regional channels on election days to see the latest prognosis and detailed results, you see numbers collected and calculated on a nicely crafted Smalltalk system written by fine developers. As far as I know, Smalltalk has been in use for predessors of this system for quite a while.

I hope Marten will continue to write about the system in General and what they learn(ed) writing and running it. While the task may sound easy at first glance, imagine a major national public broadcasting station relies on your results on Sunday afternoon at 6pm for at least 3 hours for a live programme. And imagine a few hundred or thousand interviewers sending in results on a Sunday.

You may get a feeling of how much risk you want to take when developing and running such a system….

If you are interested in Smalltalk in General, or in Gemstone/S, I encourage you to visit and/or subscribe to Marten’s Blog.

Marten found some of our old SmalltalkInspect Podcast episodes

A few years ago, Marten Feldtmann, Sebastian Heidbrink and me did our best to do some podcasting on Smalltalk. Some of our episodes were in english, some were in German. One of our highest priority decisions back then was that we can only do it if we do it with as little effort as possible.
Interestingly, things worked out quite well. Some of our episodes were not actually complete trash ;-).

We didn’t have enough time and our jobs and personal projects led us to work on different areas, so the SmalltalkInspect podcast faded into bit nirvana. One day we realized our hosting provider had deleted our account and it seemed no one had kept any backups. But Marten just found a few somewhere in the backyard in the hedges around his telescope or beneath his soldering table, not sure, he wouldn’t tell 😉

So here are some of our old episodes for you to download and listen to what the early Marten, Sebastian and Joachim did back in their youth.

I hope there is still some interesting and entertaining material in this for somebody. Listening shortly into these, I somewhat become a little nostalgic and think “what if we just…”

Instantiations to showcase new Cross-Platform OS Process Framework online

If you’ve been following the progress of VA Smalltalk over the last few years, you may have realized that the folks at Instantiations are moving real fast. It’s only been 3 years that they introduced a completely new VM with 64 bits support, they came up with beautifully crafted APIs for SMTP, IMAP and more, added support for HiDPI displays and whatnot. It’s like watching a firework going off, one rocket after the next.

Version 9.2.2 has just left the door and the folks are already showcasing a new Framework for starting external processes and treating the results of this in a cross-platform environment.

We’ve been developing Kontolino! on Windows and deploying on Linux since 2013. This works very well, and almost every version of VAST brought some improvements for VAST on Linux (which I think is important for everybody planning to move to the Cloud or the Web). One last area that forced us to always test on Linux (and fix bugs there) was the way AbtProgramStarter showed very different behavior on Windows and Linux. This will obvously come to an end.

So if you are interested in what Instantiations is going to come up with in their upcoming Version (which will have a new versioning scheme and be named) 2021, you can register for an online webinar on Thursday, July 9th, 4:00 pm CEST, in which Seth will showcase their new Cross-Platform OS Process Framework. I hear it is going to be simply great.

To register, please follow this link.

Installing EMSRV as a daemon on Ubuntu 18.04

[Addition March 15th, 2020: I was so excdited about me getting this to run that I forgot about security!. After a good night’s sleep I added a few words about running emsrv as another user than root]

Installation of VA Smalltalk on Linux has become so incredibly nice since version 9.0 all that’s needed is to download the deb (for debian or ubuntu and its forks like Mate) or rpm tar from Instantiations, unpack it (using tar -xvzf) it and run the vainstall with an install option like this:

  • vainstall manager if you want the emsrv and a fresh library
  • vainstall admintools if all you need is emsrv and emadmin
  • vainstall [standalone] for a complete installation of both the development environment and the manager library and admin tools on that machine

The basics are there…

The installation script, however, only installs all you need and creates all links and such, so that you can run emsrv by

/usr/local/VASmalltalk/9.2.1x64/bin/emsrv

…but that might not be enough. We want autorun!

For our needs, we want to have the server start emsrv on boot time without any interaction. So we needed to set emsrv up as a service on ubuntu. And it turns out this is quite easy, but things have changed quite a bit since my last description of doing the same thing on openSuSE a few years ago.

Enter systemd, which is the current system daemon on Ubuntu. This may sound scary, but in fact setting emsrv up as a service on Ubuntu is easy.

All you need to do is add a file named emsrv.service to /etc/systemd/system

[Unit]
Description=The EMSRV Va Smalltalk Library Manager
After=network.target

[Service]
Type=forking
# don't run emsrv as root!
User=emsrvuser
ExecStart=/usr/local/VASmalltalk/9.2.1x64/bin/emsrv

[Install]
WantedBy = multi-user.target

This file defines a new service (called a Unit) in systemd and tells the system that this new unit can only be started after the network is available. Take special note of the Type=forking line in the [Service] section. Without this, emsrv will not be started and will be reported as dead by systemctl status.
Please make sure you also add the Install section, otherwise you’ll get errors on startup.

Of course you can set all kinds of options for emsrv (like log file location, password etc.) by providing startup options in the ExecStart command.

Then there is one last step to perform:

systemctl enable emsrv.service

Almost there

After reboot, emsrv will be running and you cann access your mgr921.dat through emsrv. If you (like us) want the machine to act as a server in you lan, you have to make sure your firewall allows connections to port 4800 (or the port you configure on startup of emsrv). In ubuntu this is easy: ufw allow 4800. You can, of course also add a service name and use it for the firewall. I’ll leave that as an exercise for you.

Are you there?

There are several ways to find out if the emsrv service is running. The first options that come to mind are (h)top and pgrep. But also systemctl will tell you a lot about emsrv:

Security – don’t run emsrv as root

What I just showed makes the emsrv server run as root. This may not be the best of possible ideas for a number of reasons. So the next step will be to create a Linux user with limited rights to run emsrv (most importantly: limit this users access rights to folders on the machine, this user should not need too many rights and almost no access rights other than the path(s) where manager files (libraries) reside).

Remember: if you have an image that connects to this server through envy, you can always open a file browser on the server for reconnecting to a Library, exporting code etc. This way, an attacker can find out about your directory structures and possibly even create/delete files.

systemd has a User= parameter in the emsrv.service file’s [Service] section.

Seaside snippets: accepting logins from external pages

This is just another small learning about Smalltall that I hope is helpful for somebody. Nothing rocket-sciency, but a little thing that needs to be experimented with to know how it might work. I am happy to learn about improvements, better approaches or nicer solutions to the problem.

So here I am, explaining the problem. Seaside as a web server tends to feel like a monolith. Everything is controlled by Seaside and it sometimes seems to be complicated to integrate it with other technologies. This may or may not be a problem you mostly have when you come to Web programming from a classic fat client world and have had no or little exposure to web technologies before doing web development, or, to put it less negative: when you start web programming by using Seaside and Smalltalk rather than coming from a cool framework like React, Angular, VueJS or whatever.

The classical approach to having a web application which requires the user to log on before information is presented to her is to make sure the first page a visitors sees is a Login Component which only #show:’s any pages to a user once the user has entered valid credentials. The usual implementation here is to register this Login component as the root Component of your Seaside Application.

This is all fine and has been working for years.

But most web applications, at least when they are open to the public and represent a business (like Kontolino! for example), are accompanied by a web site promoting the thing. There are a few pages like “about us”, “pricing”, “imprint”, “security”, you name it. And of course there needs to be a link to the application somewhere on this page, inviting visitors to register and returning visitors to log in and start using your application.

So you may want to add a login/password dialog to this web site that directly logs teh user in to your Seaside Application. Just drop down a little form with two input fields and a login button that directly sends you to your shiny dashboard and welcome page in the application.

Since most of the times this web site is unlikely to be served by your Seaside Application but rather as static pages or some CMS like wordpress, typo3 or whatever, there is no chance to make your Seaside Application to render its Login screen inside that page.

Unless you use tricks like html embeds or iframes that request the login form from the Seaside Application. This approach, however has a few negative drawbacks, like the fact that each time the embed gets requested, Seaside will have to provide its Login form, which most likely means instantiating a WASession, and of course using CPU cycles within your Smalltalk image that should be busy serving all the connected users and not spend a certain percentage of its power to draw two input fields and a submit button.

Of course, there is a better way: let the CMS or static web pages do the job of presenting the input fields for logon and only let Seaside get into action when a user really attempts to log in.

Loose coupling is one of the strengths of the web, and this task per se is super easy for the html part of it. All you need is an html form with the two input fields for login credentials and a submit button that posts the form to your Seaside Application. Let’s have a short look at an example:

<form action="https://mySeasideApp.com:8080/?login" id="login" method="post">
  <label class="h2" form="person">Please login:</label>
  <label for="usernam">User name</label>
  <input type="text" name="username" id="username" >

  <label for="passwd">Password</label>
  <input type="password" name="passwd" id="passwd" >

  <button type="submit">Log In!</button>
</form>

There’s nothing shockingly exciting about this html snippet. Just a form whose action is you Seaside Application’s url and some parameter that indicates the user wants to login. This parameter is not even necessary, but I like to differntiate a little on the server side to present different login info on the welcome screen. Or maybe you want to log where people come from or whatever, so this ‘?login’ url parameter is probably helpful. The important thing to note here is that you should make the method a POST and also make sure you don’t accept logins if they are not POSTed. The most important thing is that in a POST request, the password will not be transported to you application as part of the URL, but in the request body instead.

So nothing fancy so far, I know. I didn’t say this will be blog post that changes your life.

The most exciting thing in this post is that you need to know how to accept logins on the Seaside server without having rendered the login from at first and not processing the credentials in a Callback.

This is where the method #initialRequest: comes in. It is implemented in WAPresenter and therefor is available in any WAComponent. Seaside sends the method to your root component whenever a request comes in and needs to be handled by a Seaside component. So the first thing to do is to implement your own #initialRequest: in your root Component. The Root component usually is the one you registered with WAAdmin when you started your Seaside app.

#initialRequest: gets invoked before anything is rendered and gets a WARequest object as a parameter. In it you will find all the info that came in in the HTTP request, like post fields, the url, the method and all that stuff.

Here, finally, is how you’d handle the login credentials in Seaside from that HTTP request:

initialRequest: request

initialRequest: request
  super initialRequest: request.
 
request at: 'login' ifPresent: [:para | ^self attemptLogonWithRequest: request].

Note: in the block we igone the parameter, since we don’t expect any url parameters to the ‘?login’ part.

anmeldenMitRequest: req

attemptLoginWithRequest: req
  | uname pw |

  "The name of the input field in the html is the name of the field in the request"
  uname := req at: 'username' ifAbsent: []. 
  pw := req fields at: 'passwd' ifAbsent: [].

  (req isPost and: [uname notNil and: [uname notNil]])
      ifTrue: [self session searchForUser: ben withPw: pw].

  self session user isNil "this is being set in #serchForUser:withPw:"
      ifFalse: [self showDashboard]
      ifTrue: [
        self
            show: (MyExistingLoginComponent new username: uname)
            onAnswer: [:success | success ifTrue: [self showDashboard]]]

That’s basically all. As I said: no rocket science. The only real information here is that you can use #initialRequest: to do some checks on the HTTP request before you do anything. You can redirect the user to certain components, check for credentials and other conditions etc.

I hope this is useful to anybody and I am looking forward to reading your comments if you know better options and have implemented neat tricks using initialRequest:.