A random walk through Mathematica 10

July 10th, 2014 | Categories: math software, mathematica | Tags:

Introduction

Mathematica 10 was released yesterday amid the usual marketing storm we’ve come to expect for a new product release from Wolfram Research. Some of the highlights of this marketing information include

Without a doubt, there is a lot of great new functionality in this release and I’ve been fortunate enough to have access to some pre-releases for a while now.  There is no chance that I can compete with the in-depth treatments given by Wolfram Research of the 700+ new functions in Mathematica 10 so I won’t try.  Instead, I’ll hop around some of the new features, settling on those that take my fancy.

Multiple Undo

I’ve been a Mathematica user since version 4 of the product which was released way back in 1999.  For the intervening 15 years, one aspect of Mathematica that always frustrated me was the fact that the undo feature could only undo the most recent action. I, along with many other users, repeatedly asked for multiple undo to be implemented but were bitterly disappointed for release after release.

Few things have united Mathematica users more than the need for multiple undo:

Finally, with version 10, our hopes and dreams have been answered and multiple undo is finally here. To be perfectly honest, THIS is the biggest reason to upgrade to version 10. Everything else is just tasty gravy.

Key new areas of functionality

Most of the things considered in this article are whimsical and only scratch the surface of what’s new in Mathematica 10.  I support Mathematica (and several other products) at the University of Manchester in the UK and so I tend to be interested in things that my users are interested in. Here’s a brief list of new functionality areas that I’ll be shouting about

  • Machine Learning I know several people who are seriously into machine learning but few of them are Mathematica users.  I’d love to know what they make of the things on offer here.
  • New image processing functions The Image Processing Toolbox is one of the most popular MATLAB toolboxes on my site. I wonder if this will help turn MATLAB-heads. I also know people in a visualisation group who may be interested in the new 3D functions on offer.
  • Nonlinear control theoryVarious people in our electrical engineering department are looking at alternatives to MATLAB for control theory. Maple/Maplesim and Scilab/Xcos are the key contenders. SystemModeler is too expensive for us to consider but the amount of control functionality built into Mathematica is useful.

Entities – a new data type for things that Mathematica knows stuff about

One of the new functions I’m excited about is GeoGraphics that pulls down map data from Wolfram’s servers and displays them in the notebook.  Obviously, I did not read the manual and so my first attempt at getting a map of the United Kingdom was

GeoGraphics["United Kingdom"]

What I got was a map of my home town, Sheffield, surrounded by a red cell border indicating an error message

Screen Shot 2014-07-10 at 00.52.53

 

The error message is “United Kingdom is not a Graphics primitive or directive.”  The practical upshot of this is that GeoGraphics is not built to take strings as arguments.  Fair enough, but why the map of Sheffield?  Well, if you call GeoGraphics[] on its own, the default action is to return a map centred on the GeoLocation found by considering your current IP address and it seems that it also does this if you send something bizarre to GeoGraphics.  In all honesty, I’d have preferred no map and a simple error message.

In order to get what I want, I have to pass an Entity that represents the UK to the GeoGraphics function.  Entities are a new data type in Mathematica 10 and, as far as I can tell, they formally represent ‘things’ that Mathematica knows about. There are several ways to create entities but here I use the new Interpreter function

Screen Shot 2014-07-10 at 01.20.15

From the above, you can see that Entities have a special new StandardForm but their InputForm looks straightforward enough. One thing to bear in mind here is that all of the above functions require a live internet connection in order to work.  For example, on thinking that I had gotten the hang of the Entity syntax, I switched off my internet connection and did

mycity = Entity["City", "Manchester"]

During evaluation of In[92]:= URLFetch::invhttp: Couldn't resolve host name. >>

During evaluation of In[92]:= WolframAlpha::conopen: Using WolframAlpha requires internet connectivity. Please check your network connection. You may need to configure your firewall program or set a proxy in the Internet Connectivity tab of the Preferences dialog. >>

Out[92]= Entity["City", "Manchester"]

So, you need an internet connection even to create entities at this most fundamental level.  Perhaps it’s for validation?  Turning the internet connection back on and re-running the above command removes the error message but the thing that’s returned isn’t in the new StandardForm:

Screen Shot 2014-07-10 at 01.35.08

If I attempt to display a map using the mycity variable, I get the map of Sheffield that I currently associate with something having gone wrong (If I’d tried this out at work,in Manchester, on the other hand, I would think it had worked perfectly!).  So, there is something very wrong with the entity I am using here – It doesn’t look right and it doesn’t work right – clearly that connection to WolframAlpha during its creation was not to do with validation (or if it was, it hasn’t helped).  I turn back to the Interpreter function:

In[102]:= mycity2 = Interpreter["City"]["Manchester"] // InputForm

Out[102]//InputForm:= Entity["City", {"Manchester", "Manchester", "UnitedKingdom"}]

So, clearly my guess at how a City entity should look was completely incorrect.  For now, I think I’m going to avoid creating Entities directly and rely on the various helper functions such as Interpreter.

What are the limitations of knowledge based computation in Mathematica?

All of the computable data resides in the Wolfram Knowledgebase which is a new name for the data store used by Wolfram Alpha, Mathematica and many other Wolfram products. In his recent blog post, Stephen Wolfram says that they’ll soon release the Wolfram Discovery Platform  which will allow large scale access to the Knowledgebase and indicated that ‘basic versions of Mathematica 10 are just set up for small-scale data access.’  I have no idea what this means and what limitations are in place and can’t find anything in the documentation.

Until I understand what limitations there might be, I find myself unwilling to use these data-centric functions for anything important.

IntervalSlider – a new control for Manipulate

I’ll never forget the first time I saw a Mathematica Manipulate – it was at the 2006 International Mathematica Symposium in Avignon when version 6 was still in beta. A Wolfram employee created a fully functional, interactive graphical user interface with just a few lines of code in about 2 minutes –I’d never seen anything like it and was seriously excited about the possibilities it represented.

8 years and 4 Mathematica versions later and we can see just how amazing this interactive functionality turned out to be. It forms the basis of the Wolfram Demonstrations project which currently has 9677 interactive demonstrations covering dozens of fields in engineering, mathematics and science.

Not long after Mathematica introduced Manipulate, the sage team introduced a similar function called interact. The interact function had something that Manipulate did not – an interval slider (see the interaction called ‘Numerical integrals with various rules’ at http://wiki.sagemath.org/interact/calculus for an example of it in use). This control allows the user to specify intervals on a single slider and is very useful in certain circumstances.

As of version 10, Mathematica has a similar control called an IntervalSlider.  Here’s some example code of it in use

Manipulate[
 pl1 = Plot[Sin[x], {x, -Pi, Pi}];
 pl2 = Plot[Sin[x], {x, range[[1]], range[[2]]}, Filling -> Axis, 
   PlotRange -> {-Pi, Pi}]; 
 inset = Inset["The Integral is approx " <> ToString[
     NumberForm[
      Integrate[Sin[x], {x, range[[1]], range[[2]]}]
      , {3, 3}, ExponentFunction -> (Null &)]], {2, -0.5}];
 Show[{pl1, pl2}, Epilog -> inset], {{range, {-1.57, 1.57}}, -3.14, 
  3.14, ControlType -> IntervalSlider, Appearance -> "Labeled"}]

and here’s the result:

intervalslider

Associations – A new kind of brackets

Mathematica 10 brings a new fundamental data type to the language, associations. As far as I can tell, these are analogous to dictionaries in Python or Julia since they consist of key,value pairs.  Since Mathematica has already used every bracket type there is, Wolfram Research have had to invent a new one for associations.

Let’s create an association called scores that links 3 people to their test results

In[1]:= scores = <|"Mike" -> 50.2, "Chris" -> 100.00, "Johnathan" -> 62.3|>

Out[1]= <|"Mike" -> 50.2, "Chris" -> 100., "Johnathan" -> 62.3|>

We can see that the Head of the scores object is Association

In[2]:= Head[scores]

Out[2]= Association

We can pull out a value by supplying a key. For example, let’s see what value is associated with “Mike”

In[3]:= scores["Mike"]

Out[3]= 50.2

All of the usual functions you expect to see for dictionary-type objects are there:-

In[4]:= Keys[scores]

Out[4]= {"Mike", "Chris", "Johnathan"}

In[5]:= Values[scores]

Out[5]= {50.2, 100., 62.3}

In[6]:= (*Show that the key "Bob" does not exist in scores*)
KeyExistsQ[scores, "Bob"]

Out[6]= False

If you ask for a key that does not exist this happens:

In[7]:= scores["Bob"]

Out[7]= Missing["KeyAbsent", "Bob"]

There’s a new function called Counts that takes a list and returns an association which counts the unique elements in the list:

In[8]:= Counts[{q, w, e, r, t, y, u, q, w, e}]

Out[8]= <|q -> 2, w -> 2, e -> 2, r -> 1, t -> 1, y -> 1, u -> 1|>

Let’s use this to find something out something interesting, such as the most used words in the classic text, Moby Dick

In[1]:= (*Import Moby Dick from Project gutenberg*)

MobyDick = 
  Import["http://www.gutenberg.org/cache/epub/2701/pg2701.txt"];
(*Split into words*)
words = StringSplit[MobyDick];
(*Convert all words to lowercase*)

words = Map[ToLowerCase[#] &, words];
(*Create an association of words and corresponding word count*)

wordcounts = Counts[words];
(*Sort the association by key value*)

wordcounts = Sort[wordcounts, Greater];
(*Show the top 10*)
wordcounts[[1 ;; 10]]

Out[6]= <|"the" -> 14413, "of" -> 6668, "and" -> 6309, "a" -> 4658, 
 "to" -> 4595, "in" -> 4116, "that" -> 2759, "his" -> 2485, 
 "it" -> 1776, "with" -> 1750|>

All told, associations are a useful addition to the Mathematica language and I’m happy to see them included.  Many existing functions have been updated to handle Associations making them a fundamental part of the language.

s/Mathematica/Wolfram Language/g

I’ve been programming in Mathematica for well over a decade but the language is no longer called ‘Mathematica’, it’s now called ‘The Wolfram Language.’  I’ll not lie to you, this grates a little but I guess I’ll just have to get used to it.  Flicking through the documentation, it seems that a global search and replace has happened and almost every occurrence of ‘Mathematica’ has been changed to ‘The Wolfram Language’

This is part of a huge marketing exercise for Wolfram Research and I guess that part of the reason for doing it is to shift the emphasis away from mathematics to general purpose programming.  I wonder if this marketing push will increase the popularity of The Wolfram Language as measured by the TIOBE index? Neither ‘Mathematica’ or ‘The Wolfram Language’ is listed in the top 100 and last time I saw more detailed results had it at number 128.

Fractal exploration

One of Mathematica’s competitors, Maple, had a new release recently which saw the inclusion of a set of fractal exploration functions. Although I found this a fun and interesting addition to the product, I did think it rather an odd thing to do.  After all, if any software vendor is stuck for functionality to implement, there is a whole host of things to do that rank higher in most user’s list of priorities than a function that plots a standard fractal.

It seems, however, that both Maplesoft and Wolfram Research have seen a market for such functionality.  Mathematica 10 comes with a set of functions for exploring the Mandelbrot and Julia sets. The Mandelbrot set alone accounts for at least 5 of Mathematica 10′s 700 new functions:- MandelbrotSetBoettcherMandelbrotSetDistanceMandelbrotSetIterationCount, MandelbrotSetMemberQ and MandelbrotSetPlot.

MandelbrotSetPlot[]

mandel10

Barcodes

I found this more fun than is reasonable!  Mathematica can generate and recognize bar codes and QR codes in various formats.  For example

BarcodeImage["www.walkingrandomly.com", "QR"]

wr-qr

Scanning the result using my mobile phone brings me right back home :)

Unit Testing

A decent unit testing framework is essential to anyone who’s planning to do serious software development. Python has had one for years, MATLAB got one in 2013a and now Mathematica has one.  This is good news!  I’ve not had chance to look at it in any detail, however. For now, I’ll simply nod in approval and send you to the documentation. Opinions welcomed.

Disappointments in Mathematica 10

There’s a lot to like in Mathematica 10 but there’s also several aspects that disappointed me

No update to RLink

Version 9 of Mathematica included integration with R which excited quite a few people I work with. Sadly, it seems that there has been no work on RLink at all between version 9 and 10.  Issues include:

  • The version of R bundled with RLink is stuck at 2.14.0 which is almost 3 years old. On Mac and Linux, it is not possible to use your own installation of R so we really are stuck with 2.14. On Windows, it is possible to use your own installation of R but CHECK THAT version 3 issue has been fixed http://mathematica.stackexchange.com/questions/27064/rlink-and-r-v3-0-1
  • It is only possible to install extra R packages on Windows. Mac and Linux users are stuck with just base R.

This lack of work on RLink really is a shame since the original release was a very nice piece of work.

If the combination of R and notebook environment is something that interests you, I think that the current best solution is to use the R magics from within the IPython notebook.

No update to CUDA/OpenCL functions

Mathematica introduced OpenCL and CUDA functionality back in version 8 but very little appears to have been done in this area since. In contrast, MATLAB has improved on its CUDA functionality (it has never supported OpenCL) every release since its introduction in 2010b and is now superb!

Accelerating computations using GPUs is a big deal at the University of Manchester (my employer) which has a GPU-club made up of around 250 researchers. Sadly, I’ll have nothing to report at the next meeting as far as Mathematica is concerned.

FinancialData is broken (and this worries me more than you might expect)

I wrote some code a while ago that used the FinancialData function and it suddenly stopped working because of some issue with the underlying data source. In short, this happens:

In[12]:= FinancialData["^FTAS", "Members"]

Out[12]= Missing["NotAvailable"]

This wouldn’t be so bad if it were not for the fact that an example given in Mathematica’s own documentation fails in exactly the same way! The documentation in both version 9 and 10 give this example:

In[1]:= FinancialData["^DJI", "Members"]

Out[1]= {"AA", "AXP", "BA", "BAC", "CAT", "CSCO", "CVX", "DD", "DIS", \
"GE", "HD", "HPQ", "IBM", "INTC", "JNJ", "JPM", "KFT", "KO", "MCD", \
"MMM", "MRK", "MSFT", "PFE", "PG", "T", "TRV", "UTX", "VZ", "WMT", \
"XOM"}

but what you actually get is

In[1]:= FinancialData["^DJI", "Members"]

Out[1]= Missing["NotAvailable"]

For me, the implications of this bug are far more reaching than a few broken examples.  Wolfram Research are making a big deal of the fact that Mathematica gives you access to computable data sets, data sets that you can just use in your code and not worry about the details.

Well, I did just as they suggest, and it broke!

Summary

I’ve had a lot of fun playing with Mathematica 10 but that’s all I’ve really done so far – play – something that’s probably obvious from my choice of topics in this article. Even through play, however, I can tell you that this is a very solid new release with some exciting new functionality. Old-time Mathematica users will want to upgrade for multiple-undo alone and people new to the system have an awful lot of toys to play with.

Looking to the future of the system, I feel excited and concerned in equal measure. There is so much new functionality on offer that it’s almost overwhelming and I love the fact that its all integrated into the core system. I’ve always been grateful of the fact that Mathematica hasn’t gone down the route of hiving functionality off into add-on products like MATLAB does with its numerous toolboxes.

My concerns center around the data and Stephen Wolfram’s comment ‘basic versions of Mathematica 10 are just set up for small-scale data access.’  What does this mean? What are the limitations and will this lead to serious users having to purchase add-ons that would effectively be data-toolboxes?

Final

Have you used Mathematica 10 yet? If so, what do you think of it? Any problems? What’s your favorite function?

Mathematica 10 links

 

  1. Szabolcs
    July 10th, 2014 at 17:20
    Reply | Quote | #1

    You should really check out the new geometrical computation feature (regions). It touches on several areas of functionality. Now we also have very fast Delaunay triangulation and mesh generation.

    Integrate[x^2, {x,y} \[Element] Disk[]]
    Integrate[x^2, {x,y} \[Element] Circle[]]
    RegionPlot[Circle[]]
    RegionPlot[Disk[]]
    Plot3D[x^2, {x,y} \[Element] Disk[]]

    PDEs can now be solves with the finite element method.

    Regarding RLink: you *can* use an external R version on Mac, and I believe Linux too. On Mac I always use an external version so I can install custom R packages. See here: https://github.com/szhorvat/IGraphR

  2. Clayton Voyles
    July 10th, 2014 at 17:59
    Reply | Quote | #2

    Thank you for your thorough and balanced review of Mathematica 10. The Mathematica 10 data access limits are based on typical Mathematica 9 usage patterns, and is intended to impact only users with large-scale data access needs. These users are part of the intended market for the upcoming Wolfram Discovery Platform.

    Thanks again, enjoy Mathematica 10!
    ~ Clayton Voyles, Wolfram Research Inc.

  3. Nguyen
    July 10th, 2014 at 18:38
    Reply | Quote | #3

    The RLink functionality is a serious disappointment to me. For Mac users, RLink won’t even allow you to load R packages. Wolfram would be better off ripping RLink out of Mathematica than leaving it in as-is.

    I have been a Mathematica user since V4 also, but I think V10 may be where I step off. My workflow has increasingly over the last two years moved away from Mathematica and become more focused on R. I have begun using Tableau for all of my data visualization needs, something I used Mathematica heavily for. Tableau has seamless integration with R.

    If I continue to rely on R and Tableau for most of my workflow, I don’t see the value in paying for the Mathematica license when it comes up for renewal early next year.

  4. esetlzn
    July 11th, 2014 at 08:29
    Reply | Quote | #4

    Great review of Mathematica 10! New atom type “Associations ” is useful.

  5. Mike Croucher
    July 11th, 2014 at 09:50
    Reply | Quote | #5

    @Clayton Voyles Thanks for the information. How can I determine what the data access limits are please?

    @Szabolcs Yes, the geometric computation feature look fantastic. I just hadn’t gotten around to doing anything with it. Your examples tell the story very succinctly I think.

    Concerning the use of an external R using RLink on Mac. Great that it works for you–I’ll give it a try. I based my comments that it can’t be done on official documentation.

  6. Szabolcs
    July 11th, 2014 at 16:25
    Reply | Quote | #6

    @Nguyen RLink works well with an external R on Mac. See my comment above on how to set it up.

    I use RLink often and it is an essential feature of Mathematica that I couldn’t do without. I am using it in ways that can’t be replaced by either R alone or Mathematica alone.

  7. Clayton Voyles
    July 11th, 2014 at 17:12
    Reply | Quote | #7

    You can find the data access limits on the pricing pages – http://www.wolfram.com/mathematica/pricing/

    It’s the number of W|A API calls/month. We are monitoring M10 usage and will adjust the limits, if needed, so that the average user never gets close to any limit.

  8. Ruben
    July 15th, 2014 at 04:55
    Reply | Quote | #8

    Regarding FinancialData, I think it would be great if Mathematica could connect with http://www.quandl.com to download the required information. I made the suggestion to Wolfram about a year ago to develop a package to connect Mathematica to Quandl but unfortunately that hasn’t happened yet.

  9. Szabolcs
    July 16th, 2014 at 17:47
    Reply | Quote | #9

    @Ruben
    Looking at their API it seems fairly easy to use. Is having an official package really warranted in this case?

    http://www.quandl.com/help/api

    Just choose the right format (Mathematica supports all of CSV, JSON, XML), piece together the URL from the right parameters, and Import.

    Import["http://www.quandl.com/api/v1/datasets/WIKI/AAPL.csv?sort_order=asc&exclude_headers=true&rows=3&trim_start=2012-11-01&trim_end=2013-11-30&column=4&collapse=quarterly&transformation=rdiff", "CSV"]

  10. ak
    July 16th, 2014 at 21:43

    First off, let me say that I really appreciate the breadth and depth of functionality on the MMA platform without getting nickel & dimed via specialized toolboxes and 10.0 is no exception.

    However, one long term disappointment for me is the lack of any improved support for HDF5 since v6. The current implementation via import/export is poor at best and the 32-bit .NetLink wrappers for the HDF libraries ( http://www.hdfgroup.org/ftp/HDF5/contrib/mathematica/ ) are slow & fickle.

    This is a notable deficit in working with large datasets >2GB in HDF5 format (for example CFD modeling files based on the cgns.org schema) with the need to read/write/update select datasets or hyperslab portions of a dataset.

    Luckily Matlink still works on MMA 10.0 so a workable option is to call any of the 64-bit Matlab HDF5 functions: ( http://www.mathworks.com/help/matlab/hdf5-files.html ). It’s kind of a kluge, but so great that it actually works! It would be very helpful to have a native MMA wrapper for the standard HDF5 functions though, especially considering the primary libraries are maintained by the HDF group.

  11. David Epstein
    July 22nd, 2014 at 19:14

    I took a look at the machine programming features, and they’re interesting if a little underdeveloped. You use “Classify” and “Predict” for supervised learning, and the program automatically picks a classifier (though it doesn’t say how). Unsupervised learning consists of a series of commands including clustering, PCA, and Hidden Markov Models, but the number of options available with each is still a bit thin. And I don’t see a number of usual suspect algorithms included, such as K-means clustering. So a step forward, and I’ll look to see if others supply code for some of the missing options.

  12. Ed Mendes
    August 25th, 2014 at 01:38

    I must say that I was really disappointed at version 10. Nothing was said about Rlink and connections to Arduino and RaspberryPi.

    Thanks to @Szabolcs (many many thanks) I can now use RLink with my own routines. BTW @Szabolcs, would you have examples on how to use data.frames from R in Mathematica?