Exciting times are coming in the next Pharo 9 release. Have you heard about Pillar? It is a markup syntax with tools to write and generate documentation, books and slides. The new Pharo 9 release will allow to format documentation with a simplified version of Pillar, called Microdown
This is how Microdown looks like with System Browser class comments:
Hi there.
Not so much of a script today. Continuing a little bit on the SLOC subject, yesterday I kind of resurrected on GitHub a package called DesignInfo. This package is based purely on Morphic (so it is possible even to run it without Roassal or Moose) and takes every package in a Pharo image, and count the lines of code (LOC, as provided by RPackage), along with other metrics like:
You can have a look with the following expressions:
DesignInfo linesOfCodeSortedMorph openInWorld. DesignInfo linesOfCodeMorph openInWorld.
A histogram is displayed where red bars represents production code and blue bars represents test code. See two screenshots of what the package provides:
Check how easy is to create an identity matrix in Pharo, in just one line: require matrix size from user and print the output:
(Array2D identity: (UIManager default request: 'Enter size of the matrix:') asInteger) asString
You might find useful the following code to convert a whole directory of image in PNG format to JPEG:
(FileSystem disk workingDirectory filesMatching: '*.png') do: [ : pngFile | pngFile asFileReference binaryReadStreamDo: [ : stream | PluginBasedJPEGReadWriter putForm: (PNGReadWriter formFromStream: stream) onFileNamed: pngFile withoutExtension , 'jpg' ] ] displayingProgress: 'Converting images to JPG...'.
Today's snippet takes a natural language text as input (a.k.a. the Corpus) where each line is considered a different document, and outputs a matrix of term documents with word mappings and frequencies for the given documents. This is also known as tf-idf, a distance metric widely used in information retrieval and provides the relevance or weight of terms in a document.
Why is not this just simple word counting?
If you increase relevance proportionally to word count, then all your query results will have words like "the" as the most relevant in the whole set of documents (or even in a single document), as it is a very common word. So you would need to decrease count for these common words, or increase count for "rare" words to get their relevance. This is where IDF (inverse document frequency) comes into play. With IDF you count documents, so you will assign low score to terms appeared in a lot of documents, then increasing the divider and decreasing relevance.
Finally, Stop words are removed and stemming is performed to reduce words with the same root.
First of all, you can install Moose-Algos (with some needed Hapax classes in a clean Pharo image by evaluating:
Metacello new configuration: 'MooseAlgos'; smalltalkhubUser: 'Moose' project: 'MooseAlgos'; version: #development; load. Gofer it smalltalkhubUser: 'GustavoSantos' project: 'Hapax'; package: 'Hapax'; package: 'Moose-Hapax-VectorSpace'; load.
Then you can execute the script:
| corpus tdm documents |
corpus := MalCorpus new.
documents := 'Julie loves me more than Linda loves me
Jane likes me more than Julie loves me'.
documents lines doWithIndex: [: doc : index |
corpus
addDocument: index asString
with: (MalTerms new
addString: doc
using: MalCamelcaseScanner;
yourself)].
corpus removeStopwords.
corpus stemAll.
tdm := HapTermDocumentMatrix on: corpus.
tdm.
Lines of code, LOC, SLOC, ELOC... one the simplest and metrics around, and we could find the method with the most LOC in the image with just one line of code (tested in Pharo 6.1):
SystemNavigation default allMethods collect: [ : m | m -> m linesOfCode ] into: (SortedCollection sortBlock: [ : a : b | a value < b value ])
For more advanced software engineering queries have a look to the cool Moose ecosystem
Let's face it, how many times you could have a mix of Natalie Portman with Smalltalk code? :) If you install a little SPARQL wrapper library in Pharo, you could for example access the Natalie's movie list querying DBPedia by writing something like the following code in the SPARQL query language:
DBPediaSearch new
setJsonFormat;
timeout: 5000;
query: 'PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>
SELECT DISTINCT ?filmName WHERE {
?film foaf:name ?filmName .
?film dbpedia-owl:starring ?actress .
?actress foaf:name ?name.
FILTER(contains(?name, "Natalie"))
FILTER(contains(?name, "Portman"))
}';
execute
To actually get only the titles you can use NeoJSON to parse the results:
((((NeoJSONReader fromString: jsonResults) at: #results) at: #bindings) collect: [ : entry | entry at: #filmName ]) collect: [ : movie | movie at: #value ]
And this is how results looks like:
Let's suppose we want to render a SVG shape described in a SVG Path. As SVG is basically XML you can grab (read: parse) the figure coordinates from the SVG path description attribute. For this we can use the XML DOM parser, Roassal and pass just the coordinates found in the "d" attribute of the "path" node, to build more complex shapes, like the following country:
| xmlTree view |
view := RTView new.
xmlTree := (XMLDOMParser onURL: 'https://www.amcharts.com/lib/3/maps/svg/belgiumHigh.svg') parseDocument firstNode.
((xmlTree findElementNamed: 'g')
nodesCollect: [ :node | | elem |
[ elem := (RTSVGPath new
path: (node attributeAt: 'd');
fillColor: Color random;
scale: 0.5) element ]
on: Error
do: [ : ex |
elem ifNotNil: [
elem model: (node attributeAt: 'title').
elem @ RTPopup.
elem ] ]])
reject: #isNil
thenDo: [ : e | view add: e ].
view open
That's basically code extracted from the Territorial library to easily render maps. Have you guessed it yet? Yes, it's Belgium!
Hi everybody. Today a simple but useful script to uncompress a ZIP file in the current image directory. Notice the #ensure: send, Smalltalk provides an very elegant way to evaluate a termination block:
| zipArchive fileRef | zipArchive := ZipArchive new. fileRef := 'myFile.zip' asFileReference. [ zipArchive readFrom: fileRef fullName; extractAllTo: FileSystem workingDirectory ] ensure: [ zipArchive close ].
Want to play and break your VM today? Try this useless saturday script just for fun:
| outStream |
outStream := FileStream newFileNamed: 'unary_sends.txt'.
Smalltalk allClasses
reject: [ : cls | (cls basicCategory = #'Kernel-Processes') or: [ cls = HashedCollection ] ]
thenDo: [ : cls |
cls class methodDictionary
select: [: sel | sel selector isUnary ]
thenCollect: [ : cm |
| result |
result := [ cls perform: cm selector ]
on: Error
do: [ :ex | (ex messageText includes: 'overridden') ifTrue: [ ex pass ] ].
[ result asString ]
on: Error
do: [ : ex2 | result := ex2 messageText ].
outStream nextPutAll: cls asString;
nextPutAll: '>>';
nextPutAll: cm selector asString;
tab;
nextPutAll: result asString; cr. ] ] .
outStream close.
I want to play a game :) The following script implements an "Is this Sequenceable?" kind of quiz. You are presented with a series of inspectors with method sources in the image, without its class name. And by looking only the source code you have to guess if the method belongs to a SequenceableCollection hierarchy or not. If you miss, you can see the class and its class hierarchy. At the end of the game, you are presenter your score:
| hits n | hits := 0. n := 3. n timesRepeat: [ | mth cls i | cls := (Collection withAllSubclasses select: #hasMethods) atRandom. mth := cls methodDict atRandom. i := GTInspector openOn: mth sourceCode. ((self confirm: 'Method belongs to a Sequenceable Collection?') = (cls isKindOf: SequenceableCollection class)) ifTrue: [ UITheme builder message: 'Good!'. hits := hits + 1 ] ifFalse: [ UITheme builder message: 'Method class is ' , cls asString , '. Class hierarchy: ' , (cls allSuperclassesExcluding: Object) asArray asString ]. i close ]. UITheme builder message: 'Your score: ' , hits asString , ' / ' , n asString.
What could be done to enhance the script? At first it would be really nice to add an option "Cannot determine with the displayed source"... (TBD) actually there are a lot of possibilities, like asking if it has any Critics, or if could be optimized, etc. Enjoy!
I' back :)
Today let's update the PSotD blog with a script to find your IP address using Zinc HTTP Components. Credits also to Sven Van Caekenberghe which helped me to figure out why Zn was getting a 403
ZnClient new systemPolicy; beOneShot; url: 'http://ifconfig.me/ip'; accept: ZnMimeType textPlain; headerAt: 'User-Agent' put: 'curl/7.54.0'; timeout: 6000; get.
Some days ago I experimented a bit to colorize a random DNA sequence given an alphabet and the desired sequence size, with a little help of BioSmalltalk. This is what I've got:
| text attributes |
text := ((BioSequence forAlphabet: BioDNAAlphabet) randomLength: 6000) sequence asText.
attributes := Array new: text size.
1 to: text size do: [ : index |
attributes at: index put: {
(TextColor color: (BioDNAAlphabet colorMap at: (text at: index))) } ].
text runs: (RunArray newFrom: attributes).
text.I built a color map for every nucleotide, based on the alphabet size. This is because in biological sequences (proteins, DNA, RNA) you have a different set of letters.
I should say I don't like the final result. Specially the lack of column alignment:
This seems to persist even trying other attributes
| text attributes |
text := ((BioSequence forAlphabet: BioDNAAlphabet) randomLength: 6000) sequence asText.
attributes := Array new: text size.
1 to: text size do: [ : index |
attributes at: index put: {
(TextColor color: (BioDNAAlphabet colorMap at: (text at: index))) .
(TextKern kern: 4) } ].
text runs: (RunArray newFrom: attributes).
text.
Maybe efforts in Bloc would make it easier for aligning text.
Suppose you want to create a directory tree at once. Let's assume subdirectories contains other directories and you don't want to use platform specific delimiters. We can do it in Pharo using the almighty #inject:into: and the FileSystem API.
| rootPath |
rootPath := Path / FileSystem disk store currentDisk / 'App1'.
#(
#('Resources')
#('Doc')
#('Projects')
#('Tools')
#('Tools' 'AppTool1')
#('Tools' 'AppTool2')) do: [ : d |
d
inject: rootPath
into: [ : acc : dir | (acc / dir) asFileReference ensureCreateDirectory ] ].
Hope you liked it
For this to work first ensure you have the MSYS2 bin directory added to the PATH environment variable. Just run the following from command line and add "c:\msys64\usr\bin\" to the end of the PATH variable:
systempropertiesadvanced
We will use ProcessWrapper, although with limited features, it works perfectly for simple tasks. And now you can run all those complex bash shell commands from Pharo :) For example to get the CPU frequencies in GHz:
| process output answer cmd |
process := ProcessWrapper new.
cmd := '"{ echo scale=2; awk ''/cpu MHz/ {print $4 "" / 1000""}'' /proc/cpuinfo; } | bc"'.
output := process
useStdout;
useStderr;
startWithShellCommand: 'set CHERE_INVOKING=1 & set MSYSTEM=MINGW64 & set MSYS2_PATH_TYPE=inherit & "c:\msys64\usr\bin\bash.exe" -c ' , cmd;
upToEnd.
^ (answer := process errorUpToEnd) isEmpty not
ifTrue: [ answer ]
ifFalse: [ output ].
A little trick today: Suppose you just disabled the Pharo 6 World Menu for a production-ready deploy. Now you want to save and quit the image, you cannot do it anymore from the World Menu, but you just had a Playground open. You can close the Playground and save the image using the following:
WorldState desktopMenuPragmaKeyword: 'noMenu'. GTPlayground allInstances anyOne window close. [ SmalltalkImage current snapshot: true andQuit: true ] fork
You can re-enable the World Menu by evaluating:
WorldState desktopMenuPragmaKeyword: 'worldMenu'.
As always, this is open to better suggestions or enhacements.
Ever wondered how to run tests in your package without using the Test Runner UI? You just need to provide the prefix of the package with tests and this piece of code will show you how to do it:
| pkgPrefix pkgSuite result |
pkgPrefix := ''.
pkgSuite := TestSuite named: 'MyApplication Tests'.
(RPackage organizer packageNames
select: [ : pkgName | pkgName beginsWith: pkgPrefix ]
thenCollect: [ : pkgName | (RPackage organizer packageNamed: pkgName) definedClasses ]) flatten
select: [ : c | (c includesBehavior: TestCase) and: [ c isAbstract not ] ]
thenCollect: [ : c | TestCase addTestsFor: c name toSuite: pkgSuite ].
result := pkgSuite run.
result printString.
This is the matching code in Pharo 6.x for the Bash one-liner to view a file with line numbers:
cat -n /path/to/file | less
The simplest way to open an viewer in Pharo is to inspect the contents of the file:
'/path/to/file' asFileReference contents.
However you wouldn't see the line numbers by default. If for some reason you also want to avoid the inspector/explorer tool, you may use the following snippet:
StandardWindow new addMorph: ( RubScrolledTextMorph new withLineNumbers; appendText: '/path/to/file' asFileReference contents) fullFrame: (0@0 corner: 1@1) asLayoutFrame; openInWorld.
You can also open a more full-featured text editor with the Rubric example class method:
RubWorkspaceExample open.
RProjectConnector is a package to access the R programming language in Pharo. The following script shows how to install and configure it in a clean image, using OSWindows to automatically get the R path installation:
Metacello new baseline: 'OSWindows'; repository: 'github://astares/Pharo-OS-Windows/src'; load.
Gofer it
smalltalkhubUser: 'VincentBlondeau' project: 'RProjectConnector';
configuration;
loadStable.
Copy R required DLL's into the VM directory:| rPath dlls |
(rPath := (WinRegistry
queryValue: 'InstallPath'
fromKey: (WinRegistryKey localMachine queryOpenSubkey: 'Software\\R-core\\R')) allButLast) notNil
ifTrue: [
dlls := (rPath asFileReference / 'bin' / 'i386') entries
select: [ : entry | entry extension = 'dll' ]
thenDo: [ : dllEntry |
dllEntry asFileReference
copyTo: Smalltalk vmDirectory asFileReference / dllEntry basename ] ].
Finally, we could test with the Iris data set:
| iris |
iris := 'eval' asREval: {RObject findVar: 'iris'}.
'plot' asREval: {
(iris at: 'Petal.Length').
(iris at: 'Petal.Width').
(#pch -> 21).
(#xlab -> 'length').
(#ylab -> 'Width').
(#bg ->((iris at: 'Species') collect: [ :value | {'red'. 'green3'. 'blue'} at: value ])).
(#main -> 'Edgar Anderson''s Iris Data')
}.
iris inspect.
Hope you find it useful.
For this one we will use a browser plug-in called CookieBro. It will allow us to import existing cookies from a web browser session into Pharo Smalltalk just using the Zinc HTTP Components, which are by default installed in the image, and NeoJSON. Let's assume then we have exported the cookies in a "cookiebro-cookies.json" file, the script translates the cookie's JSON format into a name–value pair (cookie crumb) suitable for a ZnCookieJar object. And it just requires you to enter the website you want to access:
| jar cookiesFile client | jar := ZnCookieJar new. cookiesFile := 'cookiebro-cookies.json'. (NeoJSONReader fromString: cookiesFile asFileReference) do: [ : d | jar add: (ZnCookie fromString: (String streamContents: [ : stream | d associationsDo: [ : assoc | stream nextPutAll: assoc key asString; nextPut: $=; nextPutAll: assoc value asString; nextPut: $; ]. stream skip: -1 ] )) ]. client := ZnClient new beOneShot; timeout: 5000; numberOfRetries: 0; url: 'https://...'; ifFail: [ : ex | self halt. ]. client session cookiejar: jar. client get
Of course if you have any suggestions I will be glad to read them in the comments.
