A few people asked how I wrote a webapp using Clojure with Mathematica. The solution is actually quiet simple - Clojure runs on JVM and Java interop (ability to call any Java libraries) works great. Mathematica provides a J/Link Toolkit that allows you to control the Math kernel from Java.
Why Clojure?
You could also use Java and link to Mathematica, but Clojure has the advantages of a functional and dynamic language. I won’t go into the details of Java vs. Clojure. Paul Graham made a good argument for Lisp in Revenge of the Nerds that’s definitely worth reading. Mathematica also includes a Python library for the MathLink library which does not support the newer versions of Python. You could also power Mathematica websites using webMathematica.
2+2 with Clojure and Mathematica
An equivalent of “hello world” program, following is the code performing a simple arithmetic using Mathematica. Note that to run the script, I need to force run JVM in 32-bit mode (using -d32 option) in Snow Leopard because J/Link package does not provide the 64-bit library. To run the script, all you have to do is call java and make sure to include both J/Link and clojure jars.
A Mathematica Webapp
Compojure is a micro web framework that is similar to Sinatra. It is an ideal choice to demo a stock quote webapp get latest stock price using the Mathematica’s FinancialData function.
Result:
Twitter has been just about everywhere lately. As its popularity continues to grow, how do you keep track of the expanding list of friends and followers? As more companies are using Twitter to promote their products, good tools are necessary to manage hundreds or thousands of contacts and analyze their Twitter social graph .
Twitter has an open api, and Mathematica is the best analytical tool out there. Mathematica’s image manipulation capability also makes it a very nice data visualizer. So let us plug Twitter in Mathematica, and play with the data.
In the examples, let’s use Wolfram_Alpha (my personal twitter data is not very interesting). WolframAlpha has just recently launched, and they have been actively posting news and updates with Twitter. Let’s first look at W|A’s recent tweets.
In[329]:=
In[331]:=
So here we import the user’s timeline, and transform the list of XMLObjects to a list of rules from attribute to value.
Now we can visualize the time and frequency of the user’s recent tweets in a timeline, where x-axis is the date and y-axis is the number of seconds during the day (eg. 30000s = 8:20 am).
In[334]:=
Out[336]=
After seeing Mathematica’s visualizations, we want to do analysis and play around with the friendship data. Let us load up the users Wolfram Alpha is following,
In[338]:=
Out[338]=
In[339]:=
Out[339]=
All friends’ profile data are loaded, and the size is consistent with the number on the website. To apply a filter, say look for the friends with screen name begins with “a”,
In[340]:=
Out[341]=
And put the name to the profile picture.
In[342]:=
Out[342]=
One of most important features on Wolfram Alpha is performing arithmetic and comparisons on objects. For example, try comparing between three stocks GOOG, AAPL and MSFT, or add up your nutrition value of a meal at McDonald’s. I also want to compare between a list of friends, which can be easily done by:
In[343]:=
Out[343]=
Now I’m interested to find out which friends have most followers, by sorting the friends with follower counts, and take top 20 results.
In[344]:=
Visualize of the results with a directed graph. An arrow from user A to user B indicates that user A is following user B.
In[345]:=
Out[345]=
We can also visualize the same list of users in a bubble plot. X - axis is number of followings, y - axis is number of followers, and the size of the image is determined by the ratio of followers / friends (ie, image is bigger if number of followers / number of friends is higher).
In[346]:=
Out[346]=
Because of Wolfram Alpha, most programmers probably have seen what Mathematica can do. Its applications are endless: I have been using it to analyze stock arbitrage opportunities, and we are also interested in analyzing the way people read comics on iPhone.
The examples above were over spare hours (or minutes) over last few nights just for fun. If anyone has any suggestions about Twitter / Mathematica, let me know (@kevenlin). This blog post from WolframBlog will also show you other cool things such as tweet from Mathematica.
Newest addition to the family: Damian Lin
Pair Trading is one of the ideas that I have been very interested in, but could not find right tool for analysis. Sure, most traders use Excel, but that’s simply too many copying and pasting cells. Patrick showed me the cool things he could do with Mathematica, and I thought that might be the perfect tool for stock analysis.
So what is Pair Trading? In short, it is a market-neutral strategy used by hedge funds and investment bank. Trader first identifies two stocks which prices have moved together in past. When the spread between the stocks widen for any reasons, buy the underperforming stock and short the outperforming one. When the spread eventually converge and the trader will close the positions and profit.
Mathematica has lots of impressive features for stock analysis. For example, 
One line of code will grab GOOG prices since 2006 and plot on a chart. Note that I don’t have to write parsers for stock data, because financial data is part of larger set of curated data that comes with Mathematica.
Now let’s use Mathematica on Coca-Cola(KO) and Pepsi(PEP), a classic example of a correlated pair.
As expected, the pair has similar movements in prices over lasts year. Now we plot the price of KO divided by PEP and get the following chart. (Note: Blue Curve is the KO / PEP pair ratio, Red Line in middle is mean, Blue and Yellow Lines are 1 standard deviation, and Red and Green Lines are 2 standard deviation).
The price ratio seems to oscillate around a mean. But we do see when the ratio appears to go below and above 2 standard deviation from the mean. The strategy is to execute a paired trade when the pair ratio is over +/- 2 standard deviation from mean. When the ratio reverts to mean, we will close the positions.
As shown on the chart, the price ratio is currently around +2 standard deviation, now might be a good time to SHORT KO / LONG PEP. For options trading, you would buy a KO put and buy a PEP call that expire on same time.
Trader should be aware of risk of drifting. This happens when the two correlated stock prices start to drift apart. The risk can be controlled if trader closes the pair (and take loss) if their prices do not converge within a time interval (eg. 6 months) or when pair ratio is above certain tolerable line (say +/- 3 standard deviations).
Mathematica has simplified the analysis with stats and charting library and curated finance data. Only 5 lines of code are needed to produce this analysis. It is also worth mentioning Mathematica also has awesome documentation to help you get started. If you have Mathematica, you can download the notebook.
“by doing nothing, and by doing the wrong things.” - pg via Hacker News
Family Portrait
Ingredients for great beer. Taken from Granville Island Brewery.
A classic metaphor from Taleb :
A turkey is fed for a 1000 days - every day confirms to its statistical department that the human race cares about its welfare “with increased statistical significance”. On the 1001 day, the turkey has a surprise.
“that code should be seen not as a static thing, like the answer to a math problem, but as an evolving effort to figure out the right question to be the answer to; and that it should thus be written to be easy to change.” — Paul Graham, on the one thing every software engineer should know
![Update: Years are automatically updated.
var montharray=new Array("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec")
function countup(yr,m,d){
var paststring=montharray[m-1]+" "+d+", "+yr
var today = new Date()
var difference=(Math.round((today -Date.parse(paststring))*100/(365*24*60*60*1000))/100)
document.write("Time flies. It\'s been " + difference + " years since the girls were born")
}
countup(2005,07,18)](http://25.media.tumblr.com/UZO9qYaGret4k4p6sRtCWIe5o1_500.jpg)
Update: Years are automatically updated.
One thing that bothered me with TextMate was the Find in Project function. In searching for replacement, Olivier pointed me to Ack!, which is an excellent tool to replace find+grep.
While doing so, I came across a nifty Ack bundle for Textmate. It’s REALLY FAST. Enjoy!.
Open source library for quantitative finance -
Open source framework for modeling, trading and risk management in finance. Includes Python binding. Despite the market turmoil, anyone wants to start a hedge fund?
(found via http://news.ycombinator.com/item?id=309198)
research done by mechanical turk -
brilliant! casual research by surveying random population using mechanical turk.
Some times we might want to clear memcache without restarting memcache server.
When calling Cache.get / set, we are really using a wrapper module Cache that simplify access to memcache client.
To flush memcache, call CACHE.flush_all
Contact form for iTune support -
This link is hidden so deeply, it always takes me few minutes to dig it up again.