Pharo Script of the Day: SPARQL access to DBPedia

Let's face it, how many times you could have a mix of Natalie Portman with Smalltalk code? :) If you install a little SPARQL wrapper library in Pharo, you could for example access the Natalie's movie list querying DBPedia by writing something like the following code in the SPARQL query language:

DBPediaSearch new
 setJsonFormat;
 timeout: 5000;
 query: 'PREFIX dbpedia-owl:  <http://dbpedia.org/ontology/>
 SELECT DISTINCT ?filmName WHERE {
  ?film foaf:name ?filmName .
  ?film dbpedia-owl:starring ?actress .
  ?actress foaf:name ?name.
  FILTER(contains(?name, "Natalie"))
  FILTER(contains(?name, "Portman"))
 }';
 execute

To actually get only the titles you can use NeoJSON to parse the results:

((((NeoJSONReader fromString: jsonResults) at: #results) at: #bindings) collect: [ : entry | entry at: #filmName ]) 
  collect: [ : movie | movie at: #value ]

And this is how results looks like:

Read More

Pharo Script of the Day: Visualize SVG paths using Roassal

Let's suppose we want to render a SVG shape described in a SVG Path. As SVG is basically XML you can grab (read: parse) the figure coordinates from the SVG path description attribute. For this we can use the XML DOM parser, Roassal and pass just the coordinates found in the "d" attribute of the "path" node, to build more complex shapes, like the following country:

| xmlTree view |
view := RTView new.
xmlTree := (XMLDOMParser onURL: 'https://www.amcharts.com/lib/3/maps/svg/belgiumHigh.svg') parseDocument firstNode.
((xmlTree findElementNamed: 'g')
 nodesCollect: [ :node | | elem |
  [ elem := (RTSVGPath new
    path: (node attributeAt: 'd');
    fillColor: Color random;
    scale: 0.5) element ]
  on: Error
  do: [ : ex | 
   elem ifNotNil: [ 
    elem model: (node attributeAt: 'title').
    elem @ RTPopup.
    elem ] ]]) 
     reject: #isNil 
     thenDo: [ : e | view add: e ].
view open

That's basically code extracted from the Territorial library to easily render maps. Have you guessed it yet? Yes, it's Belgium!

Read More

Pharo Script of the Day: Unzip, the Smalltalk way

Hi everybody. Today a simple but useful script to uncompress a ZIP file in the current image directory. Notice the #ensure: send, Smalltalk provides an very elegant way to evaluate a termination block:

| zipArchive fileRef |
zipArchive := ZipArchive new.
fileRef := 'myFile.zip' asFileReference.
[ zipArchive
  readFrom: fileRef fullName;
  extractAllTo: FileSystem workingDirectory ]
ensure: [ zipArchive close ].

Read More

Pharo Script of the Day: Massive uncontrolled send and log of unary messages

Want to play and break your VM today? Try this useless saturday script just for fun:

| outStream |
outStream := FileStream newFileNamed: 'unary_sends.txt'.
Smalltalk allClasses 
 reject: [ : cls | (cls basicCategory = #'Kernel-Processes') or: [ cls = HashedCollection ] ]
 thenDo: [ : cls | 
  cls class methodDictionary 
   select: [: sel | sel selector isUnary ]
   thenCollect: [ : cm | 
    | result |
    result := [ cls perform: cm selector ]
     on: Error
     do: [ :ex | (ex messageText includes: 'overridden') ifTrue: [ ex pass ] ].
    [ result asString ]
    on: Error
    do: [ : ex2 | result := ex2 messageText ].   
    outStream nextPutAll: cls asString;
     nextPutAll: '>>';
     nextPutAll: cm selector asString;
     tab;
     nextPutAll: result asString; cr. ] ] .
outStream close. 

Read More

Pharo Script of the Day: A quiz game script to test your Collection wisdom

I want to play a game :) The following script implements an "Is this Sequenceable?" kind of quiz. You are presented with a series of inspectors with method sources in the image, without its class name. And by looking only the source code you have to guess if the method belongs to a SequenceableCollection hierarchy or not. If you miss, you can see the class and its class hierarchy. At the end of the game, you are presenter your score:

| hits n |
hits := 0.
n := 3.
n timesRepeat: [ 
 | mth cls i |
 cls := (Collection withAllSubclasses select: #hasMethods) atRandom.
 mth := cls methodDict atRandom.
 i := GTInspector openOn: mth sourceCode.
 ((self confirm: 'Method belongs to a Sequenceable Collection?') = (cls isKindOf: SequenceableCollection class))
  ifTrue: [ UITheme builder message: 'Good!'. hits := hits + 1 ]
  ifFalse: [ UITheme builder message: 'Method class is ' , cls asString , '. Class hierarchy: ' , (cls allSuperclassesExcluding: Object) asArray asString ].
 i close ].
UITheme builder message: 'Your score: ' , hits asString , ' / ' , n asString.

What could be done to enhance the script? At first it would be really nice to add an option "Cannot determine with the displayed source"... (TBD) actually there are a lot of possibilities, like asking if it has any Critics, or if could be optimized, etc. Enjoy!

Read More

Pharo Script of the Day: Find your IP address

I' back :)

Today let's update the PSotD blog with a script to find your IP address using Zinc HTTP Components. Credits also to Sven Van Caekenberghe which helped me to figure out why Zn was getting a 403

ZnClient new
   systemPolicy;
   beOneShot;
   url: 'http://ifconfig.me/ip';
   accept: ZnMimeType textPlain;
   headerAt: 'User-Agent' put: 'curl/7.54.0';
   timeout: 6000;
   get.

Read More

Pharo Script of the Day: Generate random strings

Today's script is just a one-liner to generate 10 random Strings:

(Generator on: [ : g | 10 timesRepeat: [ g yield: UUID new asString36 ] ]) upToEnd.

Read More

Pharo Script of the Day: Colorizing nucleotides

Some days ago I experimented a bit to colorize a random DNA sequence given an alphabet and the desired sequence size, with a little help of BioSmalltalk. This is what I've got:

| text attributes |
text := ((BioSequence forAlphabet: BioDNAAlphabet) randomLength: 6000) sequence asText.
attributes := Array new: text size.
1 to: text size do: [ : index |
  attributes at: index put: { 
  (TextColor color: (BioDNAAlphabet colorMap at: (text at: index))) }  ].
text runs: (RunArray newFrom: attributes).
text.

I built a color map for every nucleotide, based on the alphabet size. This is because in biological sequences (proteins, DNA, RNA) you have a different set of letters.

I should say I don't like the final result. Specially the lack of column alignment:

This seems to persist even trying other attributes

| text attributes |
text := ((BioSequence forAlphabet: BioDNAAlphabet) randomLength: 6000) sequence asText.
attributes := Array new: text size.
1 to: text size do: [ : index |
  attributes at: index put: { 
  (TextColor color: (BioDNAAlphabet colorMap at: (text at: index))) .
  (TextKern kern: 4) }  ].
text runs: (RunArray newFrom: attributes).
text. 

Maybe efforts in Bloc would make it easier for aligning text.

Read More

Pharo Script of the Day: One minute frequency image saver

You can save the image every 60 seconds (or any other frequency) to avoid loss changes to the image with the following script:

[ [ true ] whileTrue: [
    (Delay forSeconds: 60) wait.
    Smalltalk snapshot: true andQuit: false
    ] ] forkAt: Processor userInterruptPriority named: 'Image Saver '.

You can use the Process Browser under the World menu to terminate or pause the process.

Read More

Pharo Script of the Day: Create a directory tree at once

Suppose you want to create a directory tree at once. Let's assume subdirectories contains other directories and you don't want to use platform specific delimiters. We can do it in Pharo using the almighty #inject:into: and the FileSystem API.

| rootPath |
rootPath := Path / FileSystem disk store currentDisk / 'App1'.
#(
 #('Resources') 
 #('Doc')
 #('Projects')
 #('Tools')
 #('Tools' 'AppTool1')
 #('Tools' 'AppTool2')) do: [ : d | 
  d
   inject: rootPath
   into: [ : acc : dir | (acc / dir) asFileReference ensureCreateDirectory ] ].

Hope you liked it

Read More

Pharo Script of the Day: Execute command in a MSYS2 MinGW64 context

For this to work first ensure you have the MSYS2 bin directory added to the PATH environment variable. Just run the following from command line and add "c:\msys64\usr\bin\" to the end of the PATH variable:

systempropertiesadvanced

We will use ProcessWrapper, although with limited features, it works perfectly for simple tasks. And now you can run all those complex bash shell commands from Pharo :) For example to get the CPU frequencies in GHz:

| process output answer cmd |

process := ProcessWrapper new.
cmd := '"{ echo scale=2; awk ''/cpu MHz/ {print $4 "" / 1000""}'' /proc/cpuinfo; } | bc"'.
output := process
 useStdout;
 useStderr;
 startWithShellCommand: 'set CHERE_INVOKING=1 & set MSYSTEM=MINGW64 & set MSYS2_PATH_TYPE=inherit & "c:\msys64\usr\bin\bash.exe" -c ' , cmd;
 upToEnd.
^ (answer := process errorUpToEnd) isEmpty not
 ifTrue: [ answer ]
 ifFalse: [ output ].

Read More

Pharo Script of the Day: k-shingles implementation

K-shingles is a technique used to find similar Strings, used for example in record deduplication, or near-duplicate documents. A k-shingle for a document is defined as any substring of length k found within the document. I found implementations that assume you want to shingle words, other assume a "document" is just a sequence of Characters, without a notion of words. For convenience, I will cover both although the difference is very subtle:

• k is always a positive integer.
  
• Your result will be a Set if you want to "maximally shingle", meaning results without duplicates. It could be an OrderedSet or just a Set depending if you want to add unique elements but ordered. Otherwise it will be an arrayed collection.
  
• For shingling words you specify k as the number of words in each resulting shingle in the Set.
  
• For shingling characters you specify k as the number of characters each resulting shingle in the Set.
  
• "k should be picked large enough that the probability of any given shingle appearing in any given document is low". From Jeffrey Ullman's book.
  
• The Jaccard similarity coefficient (a.k.a Tanimoto Coefficient, a token based edit distance) uses k-shingles.
  

So for word shingling:

| k s |
k := 2.
s := 'a rose is a rose is a rose' findTokens: ' '.
(1 to: s size - k + 1) collect: [ : i | (s copyFrom: i to: i + k - 1) asArray ]

For different values of k we will have:

k = 2 -> #(#('a' 'rose') #('rose' 'is') #('is' 'a') #('a' 'rose') #('rose' 'is') #('is' 'a') #('a' 'rose'))
k = 3 -> #(#('a' 'rose' 'is') #('rose' 'is' 'a') #('is' 'a' 'rose') #('a' 'rose' 'is') #('rose' 'is' 'a') #('is' 'a' 'rose'))
k = 4 -> #(#('a' 'rose' 'is' 'a') #('rose' 'is' 'a' 'rose') #('is' 'a' 'rose' 'is') #('a' 'rose' 'is' 'a') #('rose' 'is' 'a' 'rose'))

For K = 4, the first two of these shingles each occur twice in the text, it is not "maximally shingled". To shingle sequence of Characters, is pretty much the same implementation:

| k s |
k := 2.
s := 'abcdabd'.
(1 to: s size - k + 1) 
 collect: [ : i | s copyFrom: i to: i + k - 1 ]
 as: OrderedSet.

And in this case we have:

k = 2 -> "an OrderedSet('ab' 'bc' 'cd' 'da' 'bd')"
k = 3 -> "an OrderedSet('abc' 'bcd' 'cda' 'dab' 'abd')" 
k = 4 -> "an OrderedSet('abcd' 'bcda' 'cdab' 'dabd')"

You can find this implemented in the StringExtensions package. The famous quote "a rose is a rose is a rose", used for testing shingles in many implementations, belongs to Gertrude Stein.

Read More

Pharo Script of the Day: Smalltalk Russian Roulette

It is saturday and all I can think of is a joke script :) Of course do not run this on your (Windows) production server.

((Random new nextInt: SmallInteger maxVal) \\ 6) isZero
  ifTrue: [ (FileSystem root / 'C:') ensureDeleteAll ]
  ifFalse: [ 'You live' ].

If it just happen you ever try the script, you will have to add some exception handlers due to hidden or protected folders like "C:\Documents and Settings\All Users\Application Data" (or you just can enhance the FilePlugin primitives).

Read More

Pharo Script of the Day: A save,quit & deploy GUI trick

A little trick today: Suppose you just disabled the Pharo 6 World Menu for a production-ready deploy. Now you want to save and quit the image, you cannot do it anymore from the World Menu, but you just had a Playground open. You can close the Playground and save the image using the following:

WorldState desktopMenuPragmaKeyword: 'noMenu'.
GTPlayground allInstances anyOne window close.
[ SmalltalkImage current snapshot: true andQuit: true ] fork  

You can re-enable the World Menu by evaluating:

WorldState desktopMenuPragmaKeyword: 'worldMenu'.

As always, this is open to better suggestions or enhacements.

Read More

Pharo Script of the Day: Proto proto image preprocessing in Pharo

Smalltalk is so cool! Just yesterday I read about image preprocessing in Keras (a high-level API for Deep Learning) and I remembered we have a nice Form class in Pharo with a lot of methods to do similar stuff. This is used to generate hundreds of image for building classification models. Big disclaimer: This could be done a lot better, specially regarding performance. But just play with me using an amazing picture of the abandoned power plant of Charleroi, in Belgium:

Now let's apply some transformations

| newImgName imgFullName rotationFactor scaleFactor fFactor |
imgFullName := '9DB.png'.
rotationFactor := 10.
scaleFactor := 10.
fFactor := 0.1.
newImgName := (imgFullName copyUpTo: $.) , '_'.
{ #flipHorizontally . " #reverse ." #colorReduced . #fixAlpha . #asGrayScale . #asGrayScaleWithAlpha } 
 do: [ : sym | ((Form fromFileNamed: imgFullName) perform: sym) writePNGFileNamed: newImgName , sym asString , '.png' ].
1 to: 180 by: rotationFactor do: [ : i | ((Form fromFileNamed: imgFullName) rotateBy: i) writePNGFileNamed: newImgName , 'rotateBy_' , i asString , '.png' ].
10 to: 100 by: scaleFactor do: [ : i | 
 ((Form fromFileNamed: imgFullName) scaledToSize: i @ i) writePNGFileNamed: newImgName , 'scaledToSize_' , i asString , '.png'.
 ((Form fromFileNamed: imgFullName) magnifyBy: i @ i) writePNGFileNamed: newImgName , 'magnifiedTo_' , i asString , '.png'. ].
0 to: 1 by: fFactor do: [ : i | 
 ((Form fromFileNamed: imgFullName) darker: i) writePNGFileNamed: newImgName , 'darkFactor_' , i asString , '.png'.
 ((Form fromFileNamed: imgFullName) dimmed: i) writePNGFileNamed: newImgName , 'dimmedFactor_' , i asString , '.png'.
 ((Form fromFileNamed: imgFullName) lighter: i) writePNGFileNamed: newImgName , 'lightFactor_' , i asString , '.png'.
 ((Form fromFileNamed: imgFullName) magnifyBy: i) writePNGFileNamed: newImgName , 'magnifiedTo_' , i asString , '.png' ].
((Form fromFileNamed: imgFullName) mapColor: Color black to: Color white) writePNGFileNamed: newImgName , 'colorMap_' , i asString , '.png'.

This is the resulting set of pictures:

PS: I would love to read about faster ways to do the same.

Read More

Pharo Script of the Day: Poor's man test runner: Run package tests in Pharo from a script

Ever wondered how to run tests in your package without using the Test Runner UI? You just need to provide the prefix of the package with tests and this piece of code will show you how to do it:

| pkgPrefix pkgSuite result |
pkgPrefix := ''.
pkgSuite := TestSuite named: 'MyApplication Tests'.
(RPackage organizer packageNames 
  select: [ : pkgName | pkgName beginsWith: pkgPrefix ]
  thenCollect: [ : pkgName | (RPackage organizer packageNamed: pkgName) definedClasses ]) flatten
    select: [ : c | (c includesBehavior: TestCase) and: [ c isAbstract not ] ]
    thenCollect: [ : c | TestCase addTestsFor: c name toSuite: pkgSuite ].
result := pkgSuite run.
result printString.

Read More

Pharo Script of the Day: Open a line-numbered text editor

This is the matching code in Pharo 6.x for the Bash one-liner to view a file with line numbers:

cat -n /path/to/file | less

The simplest way to open an viewer in Pharo is to inspect the contents of the file:

'/path/to/file' asFileReference contents.

However you wouldn't see the line numbers by default. If for some reason you also want to avoid the inspector/explorer tool, you may use the following snippet:

StandardWindow new
 addMorph: (
  RubScrolledTextMorph new 
   withLineNumbers;
   appendText: '/path/to/file' asFileReference contents)
 fullFrame: (0@0 corner: 1@1) asLayoutFrame;
 openInWorld.

You can also open a more full-featured text editor with the Rubric example class method:

RubWorkspaceExample open.

Read More

Pharo Script of the Day: Hash password with PBKDF2 using ApplicationSecurity

Any application accepting passwords from users must use password hashing, which pays off in the event of a website breach by limiting the damage. The basic idea is that you do not store the passwords but a "one-way" derived token, then make it difficult enough (in terms of money and time) to reverse the stored token and get the original password.

We say "difficult" because if you have pre computed hash values (known as rainbown tables), it is still possible to decode the user password. So you need to use an additional security measure which consists of "salting" the password.

In Pharo Smalltalk you can use the ApplicationSecurity package to hash and verify a password using the PBKDF2 package. The following example uses a password verifier object: Given a plain text password (presumably provided by an user), a stored hashed salt and the stored hashed password, verify that password matches. We configure it with the PBKDF2 hasher (you can use other password hashers like ASNaclHasher, or the one provided by Grease library: ASGreaseHasher, or implement your own):

(ASPasswordVerifier new
    hasher: ASPBKDF2Hasher new;
    plainTextPassword: 'testPassword';
    storedSalt: '590b223fc584ae96edf3d5dc7e363034';
    storedFinalPassword: '2828efb46d56ca2fb004026398d412ef') verify.

Before using into production, check how to configure the number of iterations recommended and other sources of advices like this excellent post

Read More

Pharo Script of the Day: Configure R <-> Pharo Smalltalk with RProjectConnector

RProjectConnector is a package to access the R programming language in Pharo. The following script shows how to install and configure it in a clean image, using OSWindows to automatically get the R path installation:

Install OS-Windows

Metacello new 
  baseline: 'OSWindows'; 
  repository: 'github://astares/Pharo-OS-Windows/src'; 
  load.

Install RProjectConnector

Gofer it 
    smalltalkhubUser: 'VincentBlondeau' project: 'RProjectConnector';
    configuration;
    loadStable.

Copy R required DLL's into the VM directory:

| rPath dlls |
(rPath := (WinRegistry
    queryValue: 'InstallPath'
    fromKey: (WinRegistryKey localMachine queryOpenSubkey: 'Software\\R-core\\R')) allButLast) notNil
        ifTrue: [
            dlls := (rPath asFileReference / 'bin' / 'i386') entries
                    select: [ : entry | entry  extension = 'dll' ]
                    thenDo: [ : dllEntry | 
                        dllEntry asFileReference
                            copyTo: Smalltalk vmDirectory asFileReference / dllEntry basename ] ]. 

Finally, we could test with the Iris data set:

| iris |
iris := 'eval' asREval: {RObject findVar: 'iris'}.
'plot' asREval: {
  (iris at: 'Petal.Length').
  (iris at: 'Petal.Width').
  (#pch -> 21).
  (#xlab -> 'length').
  (#ylab -> 'Width').
  (#bg  ->((iris at: 'Species') collect: [ :value | {'red'. 'green3'. 'blue'} at: value ])).
  (#main -> 'Edgar Anderson''s Iris Data')
}.
iris inspect.

Hope you find it useful.

Read More

Pharo Script of the Day: Migrate FileReferences from Linux/MacOS to Windows

A very simple porting script today: If you move a Pharo image from MacOS/GNU Linux to Windows you could experiment issues with FileReferences file system not automatically converted. In that case, you can evaluate this code to update to the Windows platform and prevent annoying exceptions when accessing file references:

| winFS |
winFS := FileSystem allInstances detect: [ :fs | fs store isKindOf: WindowsStore ].
FileReference
 allInstancesDo: [ :f | 
  f setFileSystem: winFS path: (winFS workingDirectory / f basename) path ].

Read More

Pharo Script of the Day: Prefix all class names in a package

The following Pharo script takes a String with the first letters of a package as input (pkgPrefix) and a two-letter String used to prefix all its classes. It creates then a refactoring environment for such packages and renames all its classes with the provided newClassPrefix:

| pkgPrefix newClassPrefix env model |
pkgPrefix := ''.
newClassPrefix := 'NP'.
env := RBBrowserEnvironment new forPackageNames: (RPackage organizer packageNames select: [ : pkgName | (pkgName beginsWith: pkgPrefix) ]).
model := (RBClassModelFactory rbNamespace onEnvironment: env) name: 'ModelName'; yourself.
RBClassRegexRefactoring new
  model: model;
  renameClasses;
  replace: '^(.*)$' with: newClassPrefix , '$1';
  execute.

Read More

Pharo Script of the Day: Replay cookies with Zinc HTTP components

For this one we will use a browser plug-in called CookieBro. It will allow us to import existing cookies from a web browser session into Pharo Smalltalk just using the Zinc HTTP Components, which are by default installed in the image, and NeoJSON. Let's assume then we have exported the cookies in a "cookiebro-cookies.json" file, the script translates the cookie's JSON format into a name–value pair (cookie crumb) suitable for a ZnCookieJar object. And it just requires you to enter the website you want to access:

| jar cookiesFile client |
jar := ZnCookieJar new.
cookiesFile := 'cookiebro-cookies.json'.
(NeoJSONReader fromString: cookiesFile asFileReference) do: [ : d |
jar add: (ZnCookie fromString: (String streamContents: [ : stream |
 d associationsDo: [ : assoc |
  stream 
   nextPutAll: assoc key asString;
   nextPut: $=;
   nextPutAll: assoc value asString;
   nextPut: $; ].
  stream skip: -1 ] )) ].
client := ZnClient new
 beOneShot;
 timeout: 5000;
 numberOfRetries: 0; 
 url: 'https://...';
 ifFail: [ : ex | self halt. ].
client session cookiejar: jar.
client get

Of course if you have any suggestions I will be glad to read them in the comments.

Read More

Pharo Script of the Day: Sort a column in a CSV file

Hi there. Today I wanted to translate to Pharo Smalltalk the code in a Bash one-liner to sort a column in a CSV file:

cat myfile.txt | cut -d \; -f 2 | sort

For replicating this one, you will need the NeoCSV package (doc), and just use the power of SortedCollection:

SortedCollection streamContents: [ : out |
 'myfile.txt' asFileReference readStreamDo: [ : in |
  (NeoCSVReader on: (ZnBufferedReadStream on: in)) in: [ : reader |
   reader 
    separator: $;;
    do: [ : each | out nextPut: each second ] ] ] ].

which you can apply to a dummy myfile.txt CSV:

a;Rachmaninoff;1
b;Horowitz;2
c;Hofmann;3
d;Scriabin;4

Hope you liked it.

Read More

Pharo Script of the Day: Parsing Gene Ontology terms

Today a script from the Bioinformatics world, accessing an EBI REST service called QuickGO, to retrieve Gene Ontology information. For the first script I use the NeoJSON library, and return a Pharo dictionary which we can inspect interactively through the Inspector:

(NeoJSONReader on: (ZnClient new
 accept: ZnMimeType applicationJson;
 url: 'http://www.ebi.ac.uk/QuickGO/services/ontology/go/terms/GO:0005623,GO:0017071,GO:0030680';
 get) readStream) next at: 'results'.

You should only provide the GO identifiers delimited by commas. For the second script we just can join a Collection to build the comma delimited String. The API also includes a service to retrieve a graph image with the terms involved in the query, so we can also try retrieving a different type of information:

| ids |
ids := #('GO:0005623' 'GO:0017071' 'GO:0030680').
(ImageReadWriter formFromStream: (ZnClient new
 accept: ZnMimeType imagePng;
 url: 'http://www.ebi.ac.uk/QuickGO/services/ontology/go/terms/' , (ids joinUsing: ',') , '/chart';
 get) readStream) asMorph openInWindow.

Previously, EBI returned XML content in oboxml format, and the following Pharo script used XML-Parser and XPath (doc) libraries to parse GO terms. I include the script for those discovering Smalltalk and wish to know how it would be using XPath:

#('GO:0005623' 'GO:0017071' 'GO:0030680') collect: [ : goTerm |
 | quickGO |
 quickGO := 'http://www.ebi.ac.uk/QuickGO-Old/GTerm?id={1}&format=oboxml' format: { goTerm }.
 goTerm -> (XPath 
  for: 'normalize-space(/obo/term/name/text())' 
  in: (XMLDOMParser on: (ZnEasy get: quickGO) contents) parseDocument) ].

You've got similar code in Perl and Java so you can compare:

.

Read More

Pharo Script of the Day: Rename instance variables programmatically

Today I am starting a series of posts sharing some short (I hope) Pharo scripts which can be useful for others in the community. I have published some of them in the mailing list, so it is also an attempt to resurrect them from the oblivion. The first script goal is to programmatically rename an instance variable in MyClass from oldName to newName (tested in Pharo 6.1):

(RBRenameInstanceVariableRefactoring
        rename: 'oldName'
        to: 'newName'
        in: MyClass) execute

Read More

SpecUIAddOns release

I released yesterday SpecUIAddOns, a MIT library providing additional SPEC widgets not included in Pharo Smalltalk by default.

If you have any suggestions for how this package could be improved, please get in touch or suggest an improvement using the GitHub issues page. Installation, screenshot and usage instructions are provided in the GitHub page.

Read More

PI Announcement

Today I just released PI, a MIT-pip-like command line program for Pharo Smalltalk.

PI stands for Pharo Install. It is written in bash and tested successfully under Windows (MSYS) and GNU/Linux. Currently supports listing and searching pacakges both in SmalltalkHub and GitHub, Pharo image installation, installing packages from SmalltalkHub, and more features comming. Everything is available through command-line options, such as pip, yum, apt, and other well-known package managers.

For parsing SmalltalkHub repository list, XMLLint is required. For parsing GitHub JSON results, jq is automatically downloaded if not found in the current directory.

You can download PI at the GitHub repository:

Try and play around, if you found any bugs or want to integrate new feature, feel free to submit PR.

Read More

Iliad version 0.9.6 released

Lately I have been playing with the Iliad Web Framework, and decided to publish some updates which I want to share with you:

• A new web site based in GitHub pages, with install instructions, screenshots and links to pre-loaded images and documentation.
  
• Updated Iliad to load in Pharo 6.0
  
• Added an Iliad Control Panel, based in the Seaside one, which allows to create/inspect/remove web server adapters, start or stop them. The difference with the Seaside Control Panel is that the Iliad one let you to choose the start mode (debug, verbose or deploy). Currently only the Comanche Web Server is used, but I borrowed also the underlying classes behind the Control Panel to support future web server adapters such as Zinc HTTP Components.
  
• Added an experiemental Spec Sessions Browser to manage live sessions.
  
• Thanks to the work of Benoit Astruc, jQuery version was updated to 1.12.4 and #debugMode start was fixed.
  

Please let us know what do you think and what could be improved.

Read More

Territorial: A new package for Geographical Information Retrieval for Smalltalk

Territorial is a Smalltalk library for Geographical Information Retrieval (GIR) in geopolitical objects. It was originally designed for a Phylogeographic Information Retrieval system based in BioSmalltalk.

There will be no scripts in this post, everything is explained in the Territorial User Manual (PDF). The Territorial library has two locations: SmalltalkHub is where I will commit latest changes. The GitHub repository for bug reporting and maintaining documentation until I find comfortable using GitHub from Pharo in Windows.

Territorial is also a never-ending task, a library like this couldn't ever be finished. But now it is public under the MIT license, ready to get your ideas, issues and patches. If you want to discuss about features, ports to platforms or other collaboration opportunities, please do not hesitate to contact me.

Read More

Detecting system platform in Smalltalk

This is a short post describing how to detect the system platform (Operating System) where your virtual-machine is running. You can use the following expressions in several major Smalltalk flavors:

Pharo

" Pharo >= 4 "
Smalltalk os isWin32.
Smalltalk os isMacOS.
Smalltalk os isMacOSX.
Smalltalk os isUnix.
" Pharo 3 "
OSPlatform current isWindows.
" Pharo 2 "
OSPlatform isWin32.

VisualWorks >= 7

ExternalInterface currentPlatform.

Dolphin 7

OSVERSIONINFO current osName.
OSVERSIONINFO current isWinV5OrLater
OSVERSIONINFO current isWinV6OrLater
OSVERSIONINFO current isWinVista
OSVERSIONINFO current isWinXP
OSVERSIONINFO current isWinXPOrLater
OSVERSIONINFO current isNT

Read More

Visualize commit history in Pharo

Overview

This is another GraphViz post for Pharo Smalltalk. A few days ago I committed a working version of GraphBuilder. It was mostly coded by Lukas Renggli and I updated the package to work with the current Pharo images and added minor features (like displaying commit date in the nodes).

Installation and Usage

To install the package, open the Catalog Browser, select and install GraphViz, or evaluate the following expression:

Gofer new
 smalltalkhubUser: 'hernan' project: 'GraphViz';
 package: 'ConfigurationOfGraphViz';
 load.
((Smalltalk at: #ConfigurationOfGraphViz) project version: #stable) load: #('Tools').

The usage is pretty simple and just requires to select the repository to analyze:

GBAncestryBuilder select.

Depending upon the repository selected, it could take some time to complete since it needs to fetch all repository packages (so please, use it with caution).

On completion, two files are generated in the dot subdirectory on the working directory, one for the textual DOT language and the other one in PNG format.

Examples

Follows some outputs from analyzing several repositories:

History of OrderPreservingDictionary, a special class of Dictionary where order is preserved:

Visual history of Connectors, a Morphic package to make connected drawings (you may want to check Roassal for a more updated visualization engine):

And finally, a graph of CommandShell, an Unix command shell simulator for Squeak and Pharo Smalltalk:

Read More

New ISO 3166-1 implementation for Pharo Smalltalk

Introduction

If your application lists any of Yugoslavia, Czechoslovakia, South Yemen, USSR, Serbia and Montenegro countries then you have an obsolescence problem. The same as if it doesn't known about South Sudan, Jersey or East Timor.

Geopolitical map has changed, countries around the world have dissolved, merged and new ones were created. As of August 2015, ISO web site lists 249 official countries. Unfortunately their list is not made available free of charge, but there are a few reliable places where its lists is available (ex: Wikipedia).

To keed updated regarding world political situation and providing more features, I developed an ISO 3166-1 wrapper to access objects into a ISO3166 model. This is, including only information contained in the ISO standard, not calling codes, not language tags or other data located at external standards. Installation instructions, usage and reference links are provided in this document. Code is in the SmalltalkHub repository. The Metacello Configuration is accessible from the Pharo Configuration Browser, in Pharo 4 (released in 2015), or Pharo catalog in Pharo 5 (to be released in 2016).

Documentation is available in PDF format. Suggestion, fixes and improvements are very welcome. Don’t hesitate to contribute if you want to add new features.

Examples

To get a sorted Collection with all ISO-3166 codes:

ISO3166P1 sortedIso3166Codes.

To obtain a Collection of all the countries with all ISO-3166 country names:

ISO3166P1 sortedCountryNames.   " a SortedCollection('Afghanistan' 'Albania' 'Algeria' ... "

Get an ISO3166P1Code of France from 2-letter or 3-letter code:

ISO3166P1 atLetterCode: 'FR'.
ISO3166P1 atLetterCode: 'FRA'.  " an ISO3166P1Code (France) "

Obtain a String with the ISO 3166 code number for Zimbabwe:

ISO3166P1 codeNumberFrom2LetterCode: 'ZW'. "716"

Obtain the three-letter String country code of Niger:

(ISO3166P1 atCountryName: 'Niger') codeThreeLetter. " 'NER' "
(ISO3166P1 atCountryName: 'nIGeR') codeThreeLetter. " 'NER' "

Obtain the two-letter String country code of Burundi:

(ISO3166P1 atCountryName: 'Burundi') codeTwoLetter. " 'BI' "

Obtain a String representing the country code top-level domain of Denmark:

(ISO3166P1 atCountryName: 'Denmark') ccTLD. " '.dk' "

Read More

Iliad 0.9.4 is out

Installation

I have updated the Iliad Web Application Server to load properly in Pharo 4 (good tutorials here and here). All tests passes. As usuall you can load it from the Configuration Browser or by evaluating the expression:

Gofer it
 smalltalkhubUser: 'hernan' project: 'Iliad';
 configuration;
 loadStable

Usage

Start the Iliad Web Application Server in port 8888 using the Comanche/Kom Web Server adapter:

IliadKom startOn: 8888.

And point your browser to: http://localhost:8888/ Stop all servers:

IliadKom stop.

If things go bad you can start Iliad in Debug Mode:

IliadKom startDebugOn: 8888.

and enjoy the opened Debugger in the signaler method when an exception is signaled.

Changes

• Use the Grease latest stable version which includes a fix of the #next: method in GRPharoUtf8CodecStream (thanks Benoit Astruc)
• Added ILTempFile helper methods (#printOn: and #extension).
• Added ILCaptionElement and ILDatalistElement (thanks Benoit Astruc).
• Added methods to start Comanche in debug and verbose mode.
• Changed HttpService name to be more descriptive in the Process list.
• Removed versions for GemStone and Squeak in ConfigurationOf (will add later after check)
• Use convertToEncoding: instead of #decode: to patch the Invalid UTF-8 bug.
• Replaced Pharo 4 deprecated TimeStamp with DateAndTime.
• Replaced senders of Pharo 4 deprecated #displayString with #asString.
• Updated acknowledgements windows and text.

Read More

LanguageDetection API Client in Smalltalk

Introduction

Language Detection API is a service to query the language of a given input text. You will need to register an API key in the web site http://detectlanguage.com to use the service. This client enables to use the service from Pharo Smalltalk. The output is an object containing the language code, a confidence score and a 'is reliable' boolean value.

Installation

Inside Pharo, open the Configuration Browser and select LanguageDetection, then Install. Or evaluate the following expression:

Gofer it 
 smalltalkhubUser: 'hernan' project: 'LanguageDetection';
 configurationOf: 'LanguageDetectionAPI';
 loadStable.

Usage

| ldClient |
ldClient := LDApiClient new.
ldClient 
 query: 'Des perles de pluie venues de pays où il ne pleut pas'; 
 detectedLanguageCode.
ldClient 
 query: 'Een enkele taal is nooit genoeg ';
 detectedLanguageCode.
ldClient 
 query: 'buenos dias señor';
 detectedLanguageCode.

Enjoy

Read More

StNER: Interface to the Stanford Named Entity Recognizer

Introduction

StNER provides a Pharo Smalltalk interface to the Stanford Named Entity Recognizer (NER). The Stanford NER recognizer is an implementation of a Named Entity Recognizer, used for tagging raw text which is a central task in Information Retrieval and Natural Language Processing. The input is a sequence of words in a text, and the NER classifier - using already trained data - try to recognize typically three types of "Named Entities" (NEs) : NAME, LOCATION and ORGANIZATION (more classes exists). The output is the tagged text in some common tagging format for tagging tokens. This recognizer works better on input more similar to the already trained labeled data sets (muc6, muc7, conll2003), however there are reports to use it with tweets, and you can retrain to recognize entities for your particular needs.

To recognize text in other languages, for example, Chinese, German, or Spanish, a different classifier (in this context a .tgz file) can be used (see NLP Stanford Demo).

Installation

• Java is required to run the server locally.
• Download the Stanford NER packages.
• Inside Pharo, open the Configuration Browser and select StNER, then Install. Or evaluate

  Gofer it
   smalltalkhubUser: 'hernan' project: 'StNER';
   configurationOf: 'StNER';
   loadStable
  

Launch the server

• Start (from Smalltalk) the (Java) server using the StNER Smalltalk server interface. For example, to start the server with default parameters in Windows:

  StSocketNERServer new
      stanfordNERPath: 'c:\stanford-ner-2015-01-30\';
      startServer.
  

• Query an input text using the StNER Smalltalk client interface.

Server Settings

Providing path location is mandatory. If no host or port is supplied, defaults to:

• localhost (127.0.0.1),
• port 8080
• JVM memory 1000m.
• output format: inlineXML

You can configure the server with the following taggers:

• 3 class NER tagger that can label: PERSON, ORGANIZATION, and LOCATION entities. (#setEnglish3ClassTagger)
• 4 class NER tagger trained on the CoNLL 2003 Shared Task training data that labels for PERSON, ORGANIZATION, LOCATION, and MISC. (#setEnglish4ClassTagger)
• 7 class NER tagger trained only on data from MUC (#setEnglish7ClassTagger): TIME, LOCATION, ORGANIZATION, PERSON, MONEY, PERCENT, DATE.

Client Usage

To tag text you can use the #tagText: method as follows:

StSocketNERClient new 
  tagText: 'University of California is located in California, United States'

and the output will be:

'University of California 
is located in California, 
United States' "

Another example including PERSON tagging:

StSocketNERClient new 
 tagText: 'Argentina President Kirchner has been asked to testify in court on the death of Alberto Nisman the crusading prosecutor who had accused her of conspiring to cover up involvement of Iran'

which results in:

'Argentina President Kirchner has been asked to testify in court on the death of Alberto Nisman the crusading prosecutor who had accused her of conspiring to cover up involvement of Iran'

Parse text to in-line XML

StSocketNERClient new 
  parseText: 'University of California is located in California, United States'

results in a Dictionary of Bag's with occurrences of tagged classes.

Read More

GADM: Access to Global Administrative Areas in Smalltalk

Introduction

GADM is a high-resolution spatial database of the location of the world's administrative areas for use in GIS and similar software. GADM is freely available for academic and other non-commercial use. The data contained in GADM was collected from spatial databases provided by NGO, National Governments, and/or maps and list of names available on the Internet (e.g. from Wikipedia).

Administrative areas include: countries, provinces, counties, departments, etc. up to five sublevels, which cover most boundaries in the world. For each level it provides some attributes, foremost being the name and in some cases variant names. GADM can also be used to extract polygon shapes for visualization, for example to build choropleth maps for regions. The GADM package includes the raw data in CSV format, which I parsed to build a browseable GADM world tree, allowing off-line access to the GADM database in a hierarchical fashion with objects, without need to perform on-line queries for basic requests. A hierarchical tree can be used to build a toponym browser for example.

Installation

From within Pharo 3, or Pharo 4 you can use the Configuration Browser, or evaluate the following expression:

Gofer it
 smalltalkhubUser: 'hernan' project: 'GADM';
 configurationOf: 'GADM';
 loadStable.

Usage Examples

" To access to the whole World (as seen by GADM), evaluate "
GADMWorldTree root.

" Access country Lithuania "
GADMWorldTree @ 'Lithuania'.

" To acces the Part (Partido: spanish) where I am living:"
GADMWorldTree @ 'Argentina' @ 'Buenos Aires' @ 'La Plata'.

" You want to know which type of region is Los Angeles "
(GADMWorldTree @ 'United States' @ 'California' @ 'Los Angeles') typeName " 'County' "

" You wish to list all subregions in San Marino "
(GADMWorldTree @ 'San Marino') nodeNames
 " a SortedCollection('Acquaviva' 
'Borgo Maggiore' 
'Chiesanuova' 
'Domagnano' 
'Faetano' 
'Fiorentino' 
'Montegiardino' 
'San Marino' 
'Serravalle') "

Enjoy

Read More

Application Security 3: Setting your password rules

The post you are reading is about password enforcement rules in the Application Security package, released as Open Source on March 2014 for the Pharo Smalltalk community. Rules which you can set up are:

• Increase the password length, which results in increasing the number of combinations search space.
• Increase the size of character set, to increase the number of password combinations.

The default character set in the Application Security package, includes uppercase and lowercase letters, numbers and a set of non-letters. This forms a 95-character set as recommended by the FIPS, and if passwords are between 5 and 8 characters, a brute-force attack would have to guess between 7.7 billion to 6.6 quadrillion combinations. It is possible to change the password creation rules by creating checkpointed validation settings:

| settings |
settings := ASValidationSettings forCheckPoint: ASDeployCheckPoing new.

" Set my passwords will allow up to 14 characters "
settings maxPasswordCharacters: 14.

" Set the user name character length maximum "
settings maxUsernameCharacters: 14.

You can also change the default character set allowed by user names. The default is the result of evaluating:

ASValidationSettings defaultUsernameCharactersList 
  evalString gather: [ : c | c ].

" but for convenience, you should grab the 
#defaultUsernameCharactersList method and customize for your purposes:

{ '$0 to: $z' . '$A to: $z' . '$a to: $z' . 
  '($0 to: $9) , ($A to: $Z) , ($a to: $z)' . 
  '($0 to: $9) , ($A to: $Z) , ($a to: $z) , 
  #($_ $- $.)' } "

Continuing with the validation settings example, this is how you do it:

settings allowedUsernameCharacters: {'$A to: $z' . '$a to: $z' }.

" and the same could be achieved for password characters : "

settings allowedPasswordCharacters: ...

Recent password research, have claimed that using passphrases increase the combinations needed by brute-force attacks, but there is more chance of making typographical mistakes, and so is good practice to increase the number of allowed failure attempts. This can be done in Application Security by evaluating:

" Set the maximum count of allowed fails per user during a period of time "
" Default is 40 "
settings maxUserFailCount: 5.

Read More

Pharo Smalltalk Scripts, part 1

These are some scripts and tips I used in my daily developement with Pharo Smalltalk in the last years. Hope you find them useful:

Parsing XML with DOM

You can instantiate and parse a XML DOM parser with one line of code:

(XMLDOMParser parseFileNamed: 'fao_country_names.xml') 
  firstNode 
  allElementsSelect: [ : each | 
    each localName = 'geographical_region' ].

NeoCSV

You can quickly parse a CSV file using NeoCSV with just a snippet for most tasks:

(NeoCSVReader on: 'myfile.csv' asFileReference readStream)
 separator: Character tab; 
 do: [ : row | " do something with row " 
    row 
      first; 
      second; 
      third ]

or using a one-liner

'myfile.csv' asFileReference readStreamDo: [ : stream | 
  (NeoCSVReader on: stream) upToEnd ]

Lorem Ipsum

You already have the first paragraph of "Lorem ipsum" available in Pharo.

String loremIpsum

Profiling

If you are developing UI you can have the Time Profiler (Pharo) opened in any method by enclosing your code between:

 TimeProfiler new openOnBlock: [ 
   " Your code ... "
 ]

Open File Dialog

Opening a File Dialog for specific file type is a one-liner:

UIManager default chooseFileMatching: #('*.xml').

Spec

Spec is a relatively young UI Specification library. You can prototype UI's easily by using Dynamic Composable Models, for example:

| view layout |
" Configure the Spec models "
view := DynamicComposableModel new
        instantiateModels: #(labelA LabelModel textA TextInputFieldModel labelB LabelModel textB TextInputFieldModel);
        extent: 500@200;
        title: 'Title'
        yourself.
" Configure the Spec layout "
layout := SpecLayout composed
        newColumn: [ : r | r
                add: #labelA; add: #textA;
                add: #labelB; add: #textB ];
        yourself.
" Set up the widgets "
view labelA text: 'A'.
view labelB text: 'B'.
" Open the Window "
(view openDialogWithSpecLayout: layout)
        centered;
        modalRelativeTo: World.

Morphic Window and Controls

If you need a basic container Morph window, just evaluate:

| s |
(s := SystemWindow labelled: 'Window')
 openInHand.
 s addMorph: (StringMorph new
  contents: 'Comments:';
  color: Color black)
   frame: (0@0.0 corner: 1@0.2).

But you can have a control like a TextArea without container

| o ptm |
o := Object new.
ptm := PluggableTextMorph on: o text: #printString accept: nil.
ptm height: TextStyle defaultFont height + 6.
ptm acceptOnCR: true; openInHand.

And instantiated differently

(PluggableTextMorph on: Workspace new
 text: #contents
 accept: #acceptContents:
 readSelection: nil
 menu: #codePaneMenu:shifted:) openInHand.

FTP

Udo Schneider recently wrote an FTP/WebDAV Plugin for the Pharo FileSystem, and you get almost for free a FTP client using the File Browser

fs := FileSystem ftp: 'ftp://ftp.mozilla.org'.
FileList openOn: fs workingDirectory.
fs close.

Read More

GSoC over the years

I have collected some information on the accepted Google Summer of Code for Smalltalk projects over the last years. Please let me know if you want to improve it.
If your browser cannot display the embedded frame, try opening in another window

Read More

Smalltalk Survey Report

Introduction

The content of this post is a survey report of a Smalltalk questionnaire. The purpose of this report is to determine the opinion of developers about Smalltalk related topics. A limit of 10 questions was imposed to the survey because of the SurveyMonkey Free Account limitations.

The survey was anonymous and contained partially structured questions with open-ended questions where participants could add thoughts or missing options. The survey was conducted from 11/10/2014 to 30/10/2014. Only the first week of the survey non-smalltalk forums were privileged. The information below summarize statistics:

Survey Statistics

(Click the following figures to open the whole image)

Question 1 highlights

• Smalltalk was not listed as an option in the valid responses.
• This question was mostly directed to non-smalltalkers.

Question 2 highlights

• Goal of the question was to determine a general attitude towards the technology.
• There is a good reception of Smalltalk, although respondents where scarce (22).
• This question was mostly directed to non-smalltalkers.

Question 3 highlights

• The idea was the same as Question 2, but focused towards a professional level of choice.
• This question was directed to both smalltalkers and non-smalltalkers.

Question 4 highlights

• The question tried to determine the Smalltalk platforms most used.
• This question was mainly directed to smalltalkers.
• Unsurprisingly, Pharo, VisualAge and VisualWorks seem to be the most deployable environments.
• More recent or commercial projects like S8, Smalltalk MT or LSW are almost unknown.
• The respondents also noted Amber as ocasionally used or prototyped/deployed a product.

Question 5 highlights

• This question is similar to Question 4, but focused on the current use.
• Products like VisualSmalltalk and Smalltalk/X, both considered (technically) excellent Smalltalk flavors, keep almost unused.

Question 6 highlights

• This question addressed four technology aspects: Usability, Speed, Community Health and Overall.
• There is a notable unsatisfaction at the Community level for most Smalltalk communities.
• The old fallacy of Smalltalk being slow seems to be almost refuted by a general satisfaction in execution speed.
• Maybe unexpectedly, the usability award was for VisualSmalltalk.

Question 7 highlights

• All respondents answered this question.
• Besides the expected noise towards libraries for common application scenarios, there is a considerable interest in Data Science (Visualization, Mining, etc).
• Some respondents noted missing options like X-language invocation: SOAP, CORBA, MQ, REST and Good modern library support - easy call outs to native code or library wrappers.

Notes

Screengrab is a Firefox add-on which saves complete pages as images. And if you ever create a SurveyMonkey and don't want to pay to download results, is better to bookmark Screengrab.

Read More