Planet Squeak

blogs about Squeak, Pharo, Croquet and family
planet squeak - planet squeak es - planet squeak jp - planet croquet - planet squeak code - planet smalltalk

August 25, 2016

Göran Krampe

Benchmarking Spry vs Squeak

Spry is evolving quite nicely during my night hours. Focusing on performance is however still premature, but I do want to verify that I am not barking up the wrong tree, like ... in the wrong forest even. So I make trivial benchmarks from time to time, just to see and learn.

Some background on the Spry implementation may be interesting. Spry is implemented in Nim as a direct AST interpreter, it's not a JIT, in only about 2000 lines of code. It has a recursive classic "naive" design and uses a spaghetti stack of activation records, all allocated on the heap relying fully on Nim's GC to do it's work. It also relies on Nim's method dynamic dispatch in the interpreter loop for dispatching on the different AST nodes. Blocks are true closures and control structures like timesRepeat: are implemented as primitives, normally not cheating. Suffice to say, there are LOTS of things we can do to make Spry run faster!

The philosophy of implementation is to keep Spry very small and "shallow" which means we rely as much as possible on the shoulders of others. In this case, primarily Nim and it's superb features, performance and standard library.

Enough jibbering, let's do some silly damn lies - ehrm, I mean silly tests!

python | b r | b := OrderedCollection new. r := Random new. 2000000 timesRepeat: [b add: (r nextInt: 10)]. [b select: [:x | x > 8]] timeToRun

The above snippet runs in around 40 ms in latest Squeak 5.1. Nippy indeed! Ok, so in Spry then with a recently added primitive for select::

python b = [] 2000000 timesRepeat: [b add: (10 random)] [b select: [:x > 8]] timeToRun First of all, if this is the first Spry code you have seen, I hope you can tell it's Smalltalk-ish and it's even shorter. :) A few notes to make it clearer:

Spry runs this in 1000 ms, not that shabby, but of course about 25x slower than Squeak. However... I think I can double the Spry speed and if so, then we are in "can live with that-country".

Just to prove Spry is just as dynamic and cool as Smalltalk (even more so actually in many parts), we can also implement select: in Spry itself (and for the more savvy out there, yes, detect: can also be implenented using the same non local return trick as Smalltalk uses):

```python spryselect: = method [:pred result = ([] clone) self reset [self end?] whileFalse: [

n = (self next)
do pred n then: [result add: n]]

^result] ```

Without explaining that code, how fast is the same test using this variant implemented in Spry itself? 8.8 seconds, not horrible, but... I think we prefer the primitive :)

Now... let's pretend this particular case is an important bottleneck in our 20 million dollar project. We just need to be faster! The Spry strategy is then to drop down to Nim and make a primitive that does everything in Nim. Such a 7-line primitive could look like this:

```nimrod nimMeth("selectLarger8"): # evalArgInfix(spry) pulls in the receiver on the left. # We convert it to the type we expect - SeqComposite is a super type # of Block, Paren, Curly. let self = SeqComposite(evalArgInfix(spry)) # We create a new empty Block to put the selected nodes in. let returnBlok = newBlok() # We perform a regular Nim iteration. self.nodes is a Nim seq[Node]. for each in self.nodes:

# For each element we convert to IntVal, which is the node type for a Spry int.
# This will cause a catchable exception if it's not an IntVal.
if IntVal(each).value > 8:
  returnBlok.add(each)

# A primitive always returns a Node or subclass thereof, like in this case a Blok. return returnBlok ``` Then it runs in 10 ms!

Yup, it's cheating, but the 20 million dollar project wouldn't care... The thing to realize here is that its MUCH easier to cheat in Spry than it is in Squeak/Pharo. But... yes, you would need to know how to make a primitive, and as a primitive it's compiled code so you can't mess with it live, and it also presumes that each node is an IntVal. However, Spry (when I fix error handling) should gracefully handle if it isn't an IntVal, that will trigger a Nim exception that the Spry interpreter should catch.

If you have made primitives in Squeak/Pharo you know it's much more complicated. You need to take great care with allocation since the GC can move things under your feet. You must convert things to C and so on, and building the stuff is messy. Spry on the other hand shares the underlying data structures with Nim. In other words, Spry nodes are Nim objects. It's trivial to work with them, allocate new ones like newBlok() above creates a new block and so on. This is a huge deal! Recently when I started integrating libui with Spry (a pretty slick movie) I got callbacks from libui back into Spry working in like... 30 minutes of thinking. That's HUGE! Doing callbacks from C or C++ back into Squeak has been a really messy and complicated thing for YEARS. Not sure if it's any better.

Also, going pure Nim would be much faster still since it would use a seq[int] and not a seq[Node] (boxed ints) - a vast difference. So if we really wanted to work with large blocks of integers, a special such node type could easily be made that exposes primitives for it. Kinda like the FloatArray thing in Squeak, etc.

String finding

Let's look at another example where Spry actually beats Squeak. And by beat I mean really beat, by factor 4x! The test is to use findString:startingAt: in a fairly large string, to find a match, 2 million times.

python | s time | "A file with a mixed text 15712 bytes, hit is at 11499, 6 partial hits before that." s := (StandardFileStream oldFileNamed: 'string.txt') contentsOfEntireFile. time := [2000000 timesRepeat: [ s findString: 'native threads and super high performance garbage' startingAt: 12 ]] timeToRun

This snippet runs in 135 seconds in Squeak 5.1. The corresponding Spry code is:

```python

A file with a mixed text 15712 bytes, hit is at 11499, 6 partial hits before that.

s = readFile "string.txt" time = ([2000000 timesRepeat: [ s findString: "native threads and super high performance garbage" startingAt: 12 ]] timeToRun) `` Again, note how Smalltalkish the code looks - and ... you know, come on Smalltalk... reading a file? It shouldn't need to be(StandardFileStream oldFileNamed: 'string.txt') contentsOfEntireFile` for such a common and mundane task!

You gotta admit, readFile "string.txt" is nicer. But hey, says the careful reader, what the heck is that? Yes, Spry supports "prefix functions" that take arguments from the right, Rebol style. It isn't used much in Spry code, but for some things it really reads better. For example, in Spry we do echo "hey" instead of Transcript show: 'hey'. That's another thing that is overly verbose in Smalltalk and should IMHO be fixed, at least just to save poor newbies their fingers. Anyway (end of rant)....

...Spry runs that in 33 seconds! And just to get a sense for how large the primitive is in Spry, it's exactly 5 lines of code: nimrod nimMeth("findString:startingAt:"): let self = StringVal(evalArgInfix(spry)).value let sub = StringVal(evalArg(spry)).value let start = IntVal(evalArg(spry)).value newValue(find(self, sub, start))

It's quite easy to follow. We just pull in arguments and unbox them into Nim string, int, int - and then we call Nim's find and we finish by using newValue() to box the answer as a Spry IntVal again. This shows how easily - no... trivially we can map Spry behaviors to Nim library code which runs at the speed of C/C++.

Speeding up Spry

Given all this, it would still be nice to improve Spry to come say ... within 10x of Cog for general code, perhaps in this case shave it down from 1000 ms to around 300 ms. The things that I do know I should do to improve speed in general are the following:

I hope this got you interested in Spry!

August 25, 2016 10:00 PM

Torsten Bergmann

LamResearch and Pharo

A story on LamResearch and Pharo work. Nice!

by Torsten (noreply@blogger.com) at August 25, 2016 01:54 PM

Smalltalk Syntax Shirt

The Smalltalk syntax fits on a shirt as you can see here.

by Torsten (noreply@blogger.com) at August 25, 2016 01:48 PM

Eliot Miranda

We are very happy to make the following announcement

Hi, We are very happy to make the following announcement: Lam Research, a leading supplier of wafer fabrication equipment and services to the global semiconductor industry, is an experienced user of the Smalltalk programming language. Smalltalk is a key component in Lam’s software control system for a broad range of the equipment it manufactures. Tudor […]

by admin at August 25, 2016 09:04 AM

August 24, 2016

Torsten Bergmann

ESUG 2016 - First slides

First slides of ESUG 2016 are available on Slideshare.

Later all the slides, videos and all will be archived on http://www.esug.org/data/ESUG2016/

by Torsten (noreply@blogger.com) at August 24, 2016 12:51 PM

ESUG 2016 - Show us your projects

Show us your projects session from ESUG2016 is available on Periscope

by Torsten (noreply@blogger.com) at August 24, 2016 07:19 AM

August 22, 2016

Torsten Bergmann

ESUG 2016, 13th Innovation Technology Awards - MacroRecorder

MacroRecorder is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 07:36 PM

ESUG 2016, 13th Innovation Technology Awards - Kit

Kit (Programming for the rest of us) is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 07:26 PM

ESUG 2016, 13th Innovation Technology Awards - Caffeine

Caffeine is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:57 PM

ESUG 2016, 13th Innovation Technology Awards - Educational Bureau

Educational Bureau (using Phratch based on Pharo) is one of the candidates for the ESUG Technology Awards. Video is here.

by Torsten (noreply@blogger.com) at August 22, 2016 06:52 PM

ESUG 2016, 13th Innovation Technology Awards - Let it Bee

Let it Bee is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:50 PM

ESUG 2016, 13th Innovation Technology Awards - RemoteDebugger

RemoteDebugger is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:42 PM

ESUG 2016, 13th Innovation Technology Awards - OpenPonk

OpenPonk is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:41 PM

ESUG 2016, 13th Innovation Technology Awards - Woden 2 Game System

Woden 2 Game System is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:37 PM

ESUG 2016, 13th Innovation Technology Awards - PolyMath

PolyMath is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:34 PM

ESUG 2016, 13th Innovation Technology Awards - MatchTool

MatchTool is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:27 PM

ESUG 2016, 13th Innovation Technology Awards - smalltalkCI

smalltalkCI is one of the candidates for the ESUG Technology Awards

by Torsten (noreply@blogger.com) at August 22, 2016 06:26 PM

August 19, 2016

Torsten Bergmann

Cuis now runs on Spur!

Additionally to Squeak and Pharo now also Cuis Smalltalk runs on the faster Spur version of the virtual machine. Nice!

by Torsten (noreply@blogger.com) at August 19, 2016 07:36 PM

August 17, 2016

Torsten Bergmann

ESUG 2016, 13th Innovation Technology Awards

Have you written innovative Smalltalk? Then read more here.

by Torsten (noreply@blogger.com) at August 17, 2016 02:34 PM

Squeak 5.1. beta

Squeak 5.1 is in beta mode - you can already try it here: http://files.squeak.org/5.1beta/

by Torsten (noreply@blogger.com) at August 17, 2016 02:24 PM

Phratch in education in Brazil

A video about Phratch used in eduction.

by Torsten (noreply@blogger.com) at August 17, 2016 02:12 PM

August 16, 2016

Torsten Bergmann

Moose 6.0

Moose 6.0 based on Pharo 5 was released. Read more.

by Torsten (noreply@blogger.com) at August 16, 2016 07:13 AM

Release of Agile Visualization

Agile Visualization is a book about the Roassal Visualization engine. Read more here or directly jump to http://agilevisualization.com

by Torsten (noreply@blogger.com) at August 16, 2016 07:08 AM

August 10, 2016

Torsten Bergmann

Freewill

The Freewill project moves on and gets good results for ruzzle chromosomes

by Torsten (noreply@blogger.com) at August 10, 2016 07:35 AM

Marquette Camp Smalltalk September 15-18th

there is a Camp Smalltalk at Marquette. Read more.

by Torsten (noreply@blogger.com) at August 10, 2016 07:30 AM

August 09, 2016

Torsten Bergmann

SmallPOS Status

SmallPOS is a POS and accounting system for small business written using Seaside web application framework for Smalltalk. Pharo is used as platform and the Code is on SqueakSource. The license of SmallPOS is MIT.

SmallPOS itself is a framework that can be used to build POS management applications.

Looks like there are some activity now to update the framework code and there are some screenshots of a russian SmallPOS-based management system that was built with the SmallPOS framework (click to enlarge):





by Torsten (noreply@blogger.com) at August 09, 2016 08:27 AM

Modtalk

In 2014 there was a presentation about Modtalk - an MIT licensed modular compiled Smalltalk.

There is a nice webpage on https://www.modtalk.org with a copyright of 2016. So there seems to be some activity behind the scenes. But the GitHub repo is still empty. I wonder about the status of the project...

by Torsten (noreply@blogger.com) at August 09, 2016 07:58 AM

August 08, 2016

Torsten Bergmann

MatchTool implementation Tutorial

by Torsten (noreply@blogger.com) at August 08, 2016 02:24 PM

August 05, 2016

Torsten Bergmann

MatchToo for Pharo

A new tool that can help you to understand to the pattern code is working in match & rewrite rules

by Torsten (noreply@blogger.com) at August 05, 2016 06:25 AM

August 03, 2016

Torsten Bergmann

PunQLite now also moved to Pharo-NoSQL GitHub repo

Beside Voyage and MongoTalk now also the PunQLite project was moved to the "pharo-nosql" GitHub Repo as you can see in the worklog. This repo created by Esteban should host all NoSQL efforts in the Pharo community.

by Torsten (noreply@blogger.com) at August 03, 2016 12:00 PM