Russian cryptographic primitives -- the actors

Чебурашка <- Alice
Гена <- Bob
Шапокляк <- Eve

CRC32

Another piece fell into place

calc_adler32 :: String->Int->Int->Int

calc_adler32 [] a b = (b `shiftL` 16) .|. a

calc_adler32 str a b  = 

  let ap = (a + (ord $ head str)) `mod` mod_adler

      bp = (ap + b) `mod` mod_adler

  in calc_adler32 (tail str) ap bp

    

crc32 :: String -> Int

crc32 str = calc_adler32 str 1 0

*Main> crc32 "abc"
38600999

Making shingles

This is starting to get a little boring, it seems that all of my Haskell work consists of recombining lists and folding them, but either way yet another piece of the puzzle is now in place, I can now generate shingles


makeShingle :: [[a]]->[[a]]
makeShingle [] = []
makeShingle zs 
  | length zs >= 4 = (foldl(\acc z -> (acc ++ z)) [] (take 4 zs)) : (makeShingle $ drop 1 zs)
  | otherwise = []



makeShingle $ filter(/="") $ genericRev $ otherTok "this, is a string.  it is a string, like many other.  but i like it, i like it a lot!" standard_delim
["thisisastring","isastringit","astringitis","stringitisa","itisastring","isastringlike","astringlikemany","stringlikemanyother","likemanyotherbut","manyotherbuti","otherbutilike","butilikeit","ilikeiti","likeitilike","itilikeit","ilikeita","likeitalot"]

Set permutations

There is a part of my project that calls for a permuted set, that is given the original set of elements create a set of sets where each set contains a permutation of the original.  It turns out that this is more or less simple once you break the problem down into smaller parts.

There are a few algorithms to generate permutations,  I chose interleaving for it being relatively simple and straight forward.
For a set of [x,y, x] permute the first two elements
[[x, y], [y, x]]
and then interleave each set with z
[[z, x, y], [x, z, y], [x, y, z], [z, y, x], [y, z, x], [y, x, z]]

First we create a function that would take a list an a value and create permutations of that list with a value:

interleaveSet :: [a] -> a -> [[a]]
interleaveSet [] value = [[value]]
interleaveSet xs value = [ take pos xs ++ [value] ++ drop pos xs | pos <- [0..length(xs)] ]

Here are some samples:

*Main> interleaveSet [] 1
[[1]]
*Main> interleaveSet [1] 2
[[2,1],[1,2]]
*Main> interleaveSet [1,2,3,4] 5
[[5,1,2,3,4],[1,5,2,3,4],[1,2,5,3,4],[1,2,3,5,4],[1,2,3,4,5]]

So we now have a way of interleaving a set with a value, but remember that what we need to do is build this from an existing set.  What we need is something that will take the output of interleaveSet and interleave the sets created by it with the next value.  I was at first tempted to use a list comprehension once again

interleaveSets sets value = [ interleaveSet set value | set <- sets ]

but since interleaveSet outputs a [[a]] and interleaveSets will create a list of those, our return value will be [[[a]]], which is not what we want at all.


*Main> interleaveSets (interleaveSet [1,2,3,4] 5) 6
[[[6,5,1,2,3,4],[5,6,1,2,3,4],[5,1,6,2,3,4],[5,1,2,6,3,4],[5,1,2,3,6,4],[5,1,2,3,4,6]],[[6,1,5,2,3,4],[1,6,5,2,3,4],[1,5,6,2,3,4],[1,5,2,6,3,4],[1,5,2,3,6,4],[1,5,2,3,4,6]],[[6,1,2,5,3,4],[1,6,2,5,3,4],[1,2,6,5,3,4],[1,2,5,6,3,4],[1,2,5,3,6,4],[1,2,5,3,4,6]],[[6,1,2,3,5,4],[1,6,2,3,5,4],[1,2,6,3,5,4],[1,2,3,6,5,4],[1,2,3,5,6,4],[1,2,3,5,4,6]],[[6,1,2,3,4,5],[1,6,2,3,4,5],[1,2,6,3,4,5],[1,2,3,6,4,5],[1,2,3,4,6,5],[1,2,3,4,5,6]]]

Since we want to be able to feed it back to interleaveSets, this is not what we want.

This will do:

interleaveSets :: [[a]] -> a -> [[a]]
interleaveSets [] value = [[]]
interleaveSets (s:sets) value = interleaveSet s value ++ interleaveSets sets value

*Main> interleaveSets (interleaveSet [1,2,3,4] 5) 6
[[6,5,1,2,3,4],[5,6,1,2,3,4],[5,1,6,2,3,4],[5,1,2,6,3,4],[5,1,2,3,6,4],[5,1,2,3,4,6],[6,1,5,2,3,4],[1,6,5,2,3,4],[1,5,6,2,3,4],[1,5,2,6,3,4],[1,5,2,3,6,4],[1,5,2,3,4,6],[6,1,2,5,3,4],[1,6,2,5,3,4],[1,2,6,5,3,4],[1,2,5,6,3,4],[1,2,5,3,6,4],[1,2,5,3,4,6],[6,1,2,3,5,4],[1,6,2,3,5,4],[1,2,6,3,5,4],[1,2,3,6,5,4],[1,2,3,5,6,4],[1,2,3,5,4,6],[6,1,2,3,4,5],[1,6,2,3,4,5],[1,2,6,3,4,5],[1,2,3,6,4,5],[1,2,3,4,6,5],[1,2,3,4,5,6],[]]

I am not sure what that empty element at the end is, will have to deal with it later (I am sure it is a side effect of my base case)

Ah the solution is that the base case return a [] because any set ++ [] is just set

*Main> [1,2,3] ++ []
[1,2,3]

And here is the new result:
*Main>  interleaveSets (interleaveSet [1,2,3,4] 5) 6
[[6,5,1,2,3,4],[5,6,1,2,3,4],[5,1,6,2,3,4],[5,1,2,6,3,4],[5,1,2,3,6,4],[5,1,2,3,4,6],[6,1,5,2,3,4],[1,6,5,2,3,4],[1,5,6,2,3,4],[1,5,2,6,3,4],[1,5,2,3,6,4],[1,5,2,3,4,6],[6,1,2,5,3,4],[1,6,2,5,3,4],[1,2,6,5,3,4],[1,2,5,6,3,4],[1,2,5,3,6,4],[1,2,5,3,4,6],[6,1,2,3,5,4],[1,6,2,3,5,4],[1,2,6,3,5,4],[1,2,3,6,5,4],[1,2,3,5,6,4],[1,2,3,5,4,6],[6,1,2,3,4,5],[1,6,2,3,4,5],[1,2,6,3,4,5],[1,2,3,6,4,5],[1,2,3,4,6,5],[1,2,3,4,5,6]]

And finally to put it all together:

permuteSet :: [a] -> [[a]]
permuteSet [] = []
permuteSet set = foldl(\acc e -> interleaveSets acc e) seed (tail set)
  where seed = interleaveSet [] (head set)

Lets test

*Main > permuteSet [1,2,3,4]
[[4,3,2,1],[3,4,2,1],[3,2,4,1],[3,2,1,4],[4,2,3,1],[2,4,3,1],[2,3,4,1],[2,3,1,4],[4,2,1,3],[2,4,1,3],[2,1,4,3],[2,1,3,4],[4,3,1,2],[3,4,1,2],[3,1,4,2],[3,1,2,4],[4,1,3,2],[1,4,3,2],[1,3,4,2],[1,3,2,4],[4,1,2,3],[1,4,2,3],[1,2,4,3],[1,2,3,4]]

Looks about right

*Main> length $ permuteSet [1,2,3,4]

24

Number of permutations is equal to n! / (n -k)!

Where k is the number of elements from n per permutation, in our case k = n so that turns into 0! which we know is one

and the factorial of 4 is 4 * 3 * 2 *1 = 24

BASH directory recursion, scope and local variables

How quaint, I am using a bash shell script to do backups of my data, very similar to time machine if anyone is familiar.  Not sure why shell, at this point I can count on so many other things to be included, but I chose shell.  My directory recursion was failing after a few loops and I could not figure out why, until I found out that bash does not do lexical scope very well, and local variables need to be declared as local explicitly.  This way when a recursive invocation exits it's assignments will not live on.

A more readable haskell string tokenizer

Another crack and a tokenizer

otherTok :: String -> [Char] -> [[Char]]
otherTok [] _ = []
otherTok cs delim = foldl(\acc c -> if c `elem` delim then [] : acc else (head acc ++ [c]) : (tail acc) ) [] cs

*Main> otherTok "blah,blah blah. blah! blah" " ,.!"
["blah","","blah","","blah","blah","*** Exception: Prelude.head: empty list

Encountering a delimiter char produces an empty string, which I can remove later with filter, not sure what to do with the Exception.

P.S. Problem solved

otherTok :: String -> [Char] -> [[Char]]
otherTok [] _ = []
otherTok cs delim = foldl(\acc c -> if c `elem` delim then [] : acc else (head acc ++ [c]) : (tail acc) ) [""] cs

*Main> filter (/="") (otherTok "blah,blah blah. blah! blah" " ,.!")
["blah","blah","blah","blah","blah"] 

P.S.S The tokens that result are in reverse order of how the words show up in a line, also easily fixed 

rev :: [a] -> [a]
rev [] = []
rev xs = foldr(\x acc -> acc ++ [x]) [] xs 
 *Main> rev(filter(/="") (otherTok "abc,def.ghi" ",."))
["abc","def","ghi"] 

Or just use the builotin reverse and stop trying to reinvent the wheel

Mac OS finder/account slowness

I've noticed that it takes finder a good 15 seconds to startup sometimes.  None of the forums I visited had any good answers, most dealt with generic account slowness and most recommended to just create a new account.  I decided to run top and see what is running and process flags when I noticed at the top of the list mount and mount_nfs sleeping.

PID   COMMAND      %CPU TIME     #TH  #WQ  #POR #MRE RPRVT  RSHRD  RSIZE  ...

2429  mount_nfs    0.0  00:00.00 1    0    15   28   112K   288K   428K   17M    586M   2385 2428 sleeping ...

2428  mount        0.0  00:00.00 1    0    14   26   104K   284K   404K   9496K  578M   2385 2385 sleeping 

(output  discarded for brevity)

180   Finder       0.0  01:35.63 4    1    213  522  18M    51M    62M    37M    908M   180  174  sleeping 

I booted up my NFS server (which for my own reasons I do not keep constantly powered on) and tried launching finder again and this time, the window popped up with no delay. 

Shut off the NFS server, closed and reopened finder just to make sure that this delay is repeatable and reproducible.  Indeed it is and every time I open a finder window Mac OS (not sure which one of the internals) will attempt to mount_nfs.

Added the following to my NFS mount options
-dumbtimer -timeo=3

Closed the finder window clicked on the finder icon again, this time the delay was much shorter.  Hope this helps someone.

P.S. Having solved this problem does not make me a mac convert.  Figuring out why it takes finder so long to list a directory when ls only takes a split second might get me half way there.


P.S.S. Also deleting related items from /Users/$USER/Library/Preferences


Finder:
com.apple.finder.plist

Digital Photo Professional:

com.canon.Digital Photo Professional.LSSharedFileList.plist

com.canon.Digital Photo Professional.plist

helped, and now they work well even without NFS.  So the forums were right about that, just pointed to a different place when they mentioned preference cache.  So in my case I found the root cause, but resolved it the same way everyone else did.