Click here to check if anything new just came in.
January 24 2012
What has Watson been up to since Jeopardy?
It’s been about a year since the computer system IBM Watson entered a TV quiz show against two of the best people to play the game, and won.
You knew that already, right? If not, skip reading this and go watch some of the footage instead. It’ll be more interesting.
But what has been done with Watson since?
Watson still makes the news even if not as much as during Jeopardy!. And with updates on twitter, facebook and YouTube there’s a lot of info out there about the project.
With all those updates, it’d be useful to bring some of it together into an overview of what sort of work has been taking place in the last year.
I’ve said before that I’m not revealing anything new about Watson here. I’m a code monkey not a Press person. I’ll be sticking to stuff that has already been talked about publicly.
Research continues
The research didn’t stop after Jeopardy!. Research continue to push the underlying technology forwards. For example, one of the researchers is working on “improving Watson’s ability to deal with pairs of verbs that mean roughly the same thing but not exactly the same thing”.
IBM Research regularly work in collaboration with academia and the Watson project was no exception. In February, IBM announced the Universities that are contributing to the development of the project.
The University of Texas are “working to extend the capabilities of Watson… by developing a computational resource of common sense knowledge”.
The Rensselaer Polytechnic Institute are “working on a visualization component to visually explain to external audiences [Watson's] massively parallel analytics”.
The University at Albany developed an “interactive QA capability for sustained investigation” to be able to maintain the context for a series of questions and answers in a conversation, rather than take isolated individual questions like in Jeopardy!. IBM is working with UAlbany to integrate this capability into Watson for the future.
That’s not all. Other research areas being worked on with University partners such as Carnegie Mellon and MIT are outlined in the press release.
Healthcare and Watson – a little background
IBM’s main focus for the application of Watson is in healthcare.
Watson wasn’t specifically designed for Jeopardy! The quiz show was one demonstration of the architecture. Watson was given general purpose natural language texts like dictionaries and encyclopaedias, and used them to build itself a general knowledge from which it could answer quiz questions.
Post-Jeopardy!, instead of giving it general knowledge texts, we give it specialist texts (e.g. medical textbooks, journals, and research papers) and let it use them to build a specialist knowledge from which it can answer detailed medical questions.
That’s the goal: a system that uses the mountain of medical literature to build a question answering system able to be a physician’s assistant.
Perspectives on Watson: Healthcare
There are loads of statistics highlighting the need for this. For example, the amount of medical information doubles every 5 years. General practitioners spend about five hours a month reading journals – a fraction of the time that would be required to read the mountain of medical literature being published every day (One estimate suggested over 50,000 papers were published on the topic of neuroscience alone in 2010).
Watson can ingest every medical text, study and journal published, and use them all to build a knowledge from which it could answer detailed diagnostic questions.
IBM Watson: Transforming Healthcare
In March, the head of IBM Software described why healthcare was chosen as Watson’s first challenge: “healthcare was the perfect example of how Watson in its current form can be used… In healthcare, a patient defines … vague symptoms and a doctor has to consult data, experience and literature to figure out the cause. Tests are ordered. It’s complex…” In many ways, the medical approach of differential diagnosis is a good analogy for how Watson works, so bringing this to healthcare makes sense.
To make this vision a reality, IBM needs partners who understand healthcare, so a number of partners were announced in 2011 who understand the problems that the medical profession deal with and what workflows would work with them.
Columbia University are helping identify issues in the practice of medicine where Watson may be able to contribute, and the University of Maryland are working to identify the best way Watson could interact with doctors.
Columbia University Medical Centre
About a year before Watson went on Jeopardy!, Herbert Chase – professor of clinical medicine at Columbia University – and two of his students started performing a series of tests for Watson’s ability to handle the medical domain.
They asked Watson thousands of medical diagnostic questions, and manually reviewed it’s answers. When Watson got a question wrong, they tried to work out how close it was and why, and reported their findings to the Watson team.
In May last year, a demonstration was given of Watson’s ability to assist with medical diagnoses. “Watson was gradually given information about a fictional patient with an eye problem. As more clues were unveiled — blurred vision, family history of arthritis, Connecticut residence — Watson’s suggested diagnoses evolved from uveitis to Behcet’s disease to Lyme disease. It gave the final diagnosis a 73 percent confidence rating.”
The demonstration isn’t online, but a shortened, similar walkthrough was used in a Watson keynote presentation last November.
It’s a compelling demo, showing how a conversational, interactive question-answering system could build up a picture of what is wrong with a patient to deliver a suggested diagnosis based on a wide variety of factors, without being limited to the common answers.
Professor Chase has spoken of similar examples he has tried on Watson, such as “a tough case he had experienced as a young doctor: a woman in her thirties with severe muscle weakness, who had blood tests indicating a low level of phosphate and elevated alkaline phosphatase, an enzyme. Watson’s top suggestions were hyperparathyroidism and rickets. It also flagged the possibility of a rare form of rickets that is vitamin D resistant—which the woman indeed had.”
They’re also considering the impact of other sources of data Watson could use, such as anecdotal evidence. What would be the effect of including discussions from blogs, message boards and other online forums?
University of Maryland School of Medicine
At a similar time, about a year before Jeopardy!, IBM started working with Dr Eliot Siegel, professor at the University of Maryland.
They suggested what would be appropriate medical literature for Watson to learn from – essentially preparing a curriculum suitable for a medical Watson, such as all of Medline, PubMed, and dozens of selected textbooks.
They provided Watson with access to (anonymised) medical records, helping to teach Watson about unusual cases with rare diseases or unexpected symptoms, and how to match what it knows about diagnostics with the experiences of procedures, treatments and outcomes that follow.
There was a great interview with Dr Siegel in March, which I recommend watching. He has a fascinating perspective on the work being done to create a medical Watson system.
Watson Computer Comes to the University of Maryland
Maryland is a med school – they teach med students. In many ways, he describes their work with Watson in a similar way. They’re training Watson as a med student, albeit a new type of student. Siegel even talks about a need for a new branch of the medical school focused at training software rather than people.
It’s not about manually programming Watson with medical information. In fact, Dr Siegel highlights that this sort of work wasn’t possible before Watson because of Watson’s unique ability to ingest information in natural language by itself.
They identified what a medical Watson system would need to know, and are working out what it needs to be taught in order to get there. They also test Watson to measure it’s progress, asking it every question from board exams, and “the clinicopathological (CPC) puzzlers” that appear in each issue of the New England Journal of Medicine.
Siegel warns that teaching Watson may take a similar amount of time to teaching a human med student. In May, he suggested that they would be three to five years from a pilot test with doctors, and that widespread use of Watson as a diagnosis tool would be maybe 8 to 10 years away. This view was echoed by the general manager of IBM Watson Systems who said that “realizing the full value of Watson will take five, seven, or maybe as many as 10 years“.
In an interview in February, Dr Siegel outlined the vision for how a trained Watson system could help:
- Read electronic medical records, create summaries and call out what’s important
- Read all of the literature in a doctor’s specialty
- Check for drug interactions
- Comb through clinical data and suggest possible diagnoses and potential treatments
- Advise a doctor in real time when they’re meeting with a patient
As I wrote about recently, he is not talking about replacing doctors, rather a tool for use by doctors.
WellPoint
In September, a partnership was announced with WellPoint, a health insurance company in the US. WellPoint is working with IBM to build Watson systems for healthcare.
A WellPoint executive gave a presentation in November outlining why they are involved. They think Watson could help in the face of rising costs of chronic medical treatment, the need to make healthcare affordable, and the under-utilisation of evidence-based medicine.
As a massive healthcare provider in the United States and the largest provider in an association providing health insurance to over 100 million people, WellPoint have access to a massive amount of medical data, which is a critical requirement for Watson’s development.
One of their first tasks was preparing their data, integrating and collecting data from several disparate sources, such as reference texts, results from clinical trials and studies, and historical medical records.
Having access to a lot of data is an asset, but with so much of it in unstructured written observations and doctors reports, they aren’t able to do as much with it as they would like. Watson could help them to improve the flow of data between the providers that they support.
Ultimately, their goal is similar to other partners – to work out how to train Watson for medical. Their focus is on it’s potential for evidence-based diagnosis and treatment.
They’re working with IBM to identify how Watson can ingest domain-specific sources, and are performing training with their real (anonymised) case data.
They’re also comparing Watson’s performance with existing clinical decision support tools. Software-based clinical decision support systems are not new, and so it’s important to prove that Watson really is something different.
She described the phases of this, with ingesting of medical literature as like Watson being at medical school, and the training phase as like Watson doing a medical residency.
Transforming Health Care Through Data
They also have a number of other areas where they plan to help.
To make Watson’s technical innovation real, they see a need for other areas of innovation: innovation in business processes, innovation in payment methods, and working out the realities of ownership, intellectual property, privacy, permission and other policy issues for a system that uses very personal data from a wide variety of sources.
What about security, what about HIPA compliance, what about liability, where can anonymised data be used, and where can’t it? These are also the sorts of areas where WellPoint think they can help.
They’re going to perform two pilots.
One pilot will be using Watson as a clinical decision support tool for use by WellPoint’s nurses responsible for managing complex patient cases and reviewing treatment requests from medical providers.
The pilot will be to see if Watson could improve the efficiency of clinical review of complex cases – helping them to make more streamlined authorisation of services, and include more evidence and explanation in their approval responses. It will also be to see if they can make more personalised and patient-specific decisions.
The second pilot is to trial the use of Watson in oncology practices allowing doctors to access Watson through a web-based platform to support the evidence-based diagnosis, treatment, and coordination of care of cancer patients – in particular, breast, lung and colon cancer.
The decision to focus on oncology is described as being because of how complex it is as a field. Essentially, to be a useful validation of Watson, it needs to be a complex field with high level of variability between treatments.
In addition, it’s an expensive area: cancer treatment costs are growing faster than other areas, so the potential benefits of Watson’s ability to make healthcare more affordable to provide are even greater.
After cancer, diabetes and cardiology are the next diseases being thought of as suitable projects.
WellPoint and Cedars-Sinai Medical Center
The first cancer clinic chosen by IBM and WellPoint for this second pilot was Cedars-Sinai‘s cancer centre. Their historical data on cancer and current clinical records were ingested into a version of Watson being hosted by WellPoint. Cedars-Sinai doctors are piloting the use of Watson as an adviser for oncologists and providing feedback and advice on the development of applications built on the Watson system.
“Watson Advisers” is a term increasingly being used for solutions built on Watson. The Watson Oncology Adviser is the highest profile one, and with Cedars-Sinai and Wellpoint, it is being taught about cancer using actual cases that have been solved, as well as what can be found in medical references.
Understanding speech
For Jeopardy!, Watson used text-to-speech to give it’s answers. It didn’t use speech-to-text to get questions. It received questions as text files.
A future Watson could receive questions verbally. In February, IBM announced a partnership with the speech technology vendor Nuance Communications.
This isn’t the first time IBM and Nuance have worked together – in 2009, a joint initiative was announced. Nuance help bring IBM speech research to the market. Later in 2009, Nuance bought several patents relating to speech recognition from IBM.
IBM and Nuance have worked together in healthcare, too. In 2010, IBM and Nuance announced a partnership in speech recognition for getting doctor’s dictated text into the structured fields of electronic health records.
Nuance have a history of bringing speech recognition to healthcare, with their specialist software for understanding clinical language. Integrating this capability with Watson brings the potential of a Watson system that can understand what doctors say. Nuance described working with Watson as a next logical step for them, taking them from “recognizing what was said to understanding the intent”.
IBM and Nuance started talking about hoping to prepare commercial tools from this collaboration in 18 – 24 months.
Ready for Watson
When Watson solutions become widely available, it will not be a trivial system for businesses to start using. There will be a need to identify and prepare data for Watson to ingest in order to build knowledge. There will be a need to perform training and testing – asking enough questions with known answers for Watson to build it’s machine learning models, and for administrators to validate the accuracy.
As a result, even though Watson isn’t commercially available today, there are customers starting to think about how they might prepare for it.
In October, IBM announced the first “Ready for Watson” offering. Ready for Watson offers companies the chance to use technologies with elements that are related to Watson and are commercially ready today. More importantly, they provide assurance that they’ll be compatible with future Watson solutions.
Customers can invest time in getting going with analytics, and get the benefits this can bring today. And in the longer-term, this effort may serve as a on-ramp to a future Watson solution.
The first Ready for Watson offering to be announced was IBM Content and Predictive Analytics (ICPA) for Healthcare
The content analytics uses the same type of natural language processing as Watson, helping to extract meaning from unstructured medical information. The predictive analytics can be used to identify deviations and root causes, and predict the probability of outcomes.
Seton Healthcare Family
One of the first users of ICPA was Seton Healthcare Family, a network of medical facilities throughout central Texas. Seton are using ICPA to identify root causes of hospital readmissions, and predict ways to decrease preventable multiple hospital visits (in particular, in patients with congestive heart failure).
This isn’t Watson, and it isn’t doing question answering. But it is using some of Watson’s techniques to perform content analysis on the 80% of medical data which is in unstructured natural language, and doing deep content analysis on that to derive useful, actionable knowledge.
In the first four months of use, it’s already produced promising results identifying possible factors that Seton are now investigating.
Call centres, retail and sales
Other industries are being considered for Watson. One of these is helping call centres to cope with the millions of questions they receive, and come up with answers more quickly and effectively than they can with flow charts or FAQs.
Perspectives on Watson: Customer Service
In February, “IBM executives said they are in discussions with a major consumer electronics retailer to develop a version of Watson… able to interact… on a variety of subjects like buying decisions and technical support“.
In March, the head of IBM Software talked about the potential for small “baby Watson’s for specific institutional tasks like call-centers“.
The thinking is that Watson could be a tool to help call-centre staff get answers for their clients while still on the phone. Unfortunately, some of this was misinterpreted by articles talking about Watson being used to directly respond to callers or replacing telemarketers.
However, in all of the discussion that this misunderstanding prompted, the potential for Watson as a tool for use by sales or call-centre staff did get people thinking.
No announcements have been made about Watson for retail, however IBM has talked about a future vision of Watson’s question answering ability available to shop assistants through an in-store kiosk or mobile tablet.
Back to Jeopardy!
In terms of development, Jeopardy! is behind Watson. No new work is being done to get better at game shows.
The stage that was used for the Watson Jeopardy! challenge was finally disassembled in October, with some of the pieces going to the Smithsonian Museum.
However, from time to time, Watson was tempted back into the game. At the end of February, Watson played an exhibition match with five US Congress members at an event focusing on the importance of IT for the US economy and the need for greater focus on education in science.
New Jersey Congressman Rush Holt beat Watson. Perhaps it was fortunate that this wasn’t the match that got televised.
Visiting universities
In a similar vein, Watson visited universities and colleges to compete in Jeopardy! matches against student teams. In October, Watson was the showcase of a symposium with Harvard Business School and MIT’s School of Management.
This time, at least, Watson won.
This was part of a university tour, following visits to Carnegie Mellon and the University of Pittsburgh, with later visits to universities like Stanford and UC Berkeley in November. There was as much focus on getting business school students thinking about the potential of technology as for the science students.
That said, these events came with some downside. The University games were run using a travelling version of Watson. A large number of Jeopardy! questions are run through “real” Watson before the event, and some custom code collects both Watson’s answer and the amount of time it took to come up with it.
In other words, we know what Watson would’ve answered to that question, and how quickly it would’ve been able to buzz in. This is then used by the travelling Watson simulator. It doesn’t need a cluster of servers to run, as it just has to buzz in after the pre-defined amount of time with the pre-defined answer.
In terms of “competing against Watson”, it is equivalent to what would’ve happened if the rack of servers had been dragged to the events. However, it’s tricky to explain, and was not well described in some press articles, leading to some comments like:
“Waiiit a minute. All these months, news on these Watson appearances had me believing he (she, it) was conjuring answers in real-time … Yet this story reports … Watson had already come up with answers to the questions prior to [the event]… Am I to understand Watson gets his (her, its) questions ahead of time?!” and “Sounds like IBM cheated”
Productisation
Lastly, development. There is development work needed any time you take a research project and turn it into a product.
There is the obvious work like internationalisation, serviceability and so on, but that’s not all.
Some is architectural, as a system created for use for short periods answering questions from a single user (i.e. Alex Trebek) is productised into a system capable of running for long periods with multiple users. And made robust enough to cope with administrators who aren’t the PhD’s who invented it and will do things wrong.
Some is usability – creating tooling necessary to turn a complex Research project into a system that can be administered, configured and used without a PhD or machine-learning expertise.
Watson on Jeopardy! was one demonstration of the DeepQA architecture. Creating a platform that can be adapted to take on other domains means working out the steps necessary to do that domain adaptation, and building procedures and tools to perform and verify it.
I couldn’t find any articles talking about this aspect of the work. It’s not terribly sexy, but it’s the area where I work so I think it’s interesting enough to mention.
And more
That’s enough for now.
Is this a complete list? Nope. For one thing, I did virtually no research. I’ve written about stuff that I can remember. There are almost certainly things I’ve forgotten.
And there are bound to be projects I’m not aware of. There is some work that I’m aware of but haven’t seen press releases for, so have to keep quiet on for now.
But this is already long enough. My aim was to give a taste of the sort of stuff that Watson has been up to.
What happens next? Watch this space.
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
January 21 2012
“FAITH”
Or, “what happens when we’re in the park with a digital camera and some time to kill”. 
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
January 14 2012
Generating a list of REST APIs in JAX-RS
Overview
Using Java Reflection to generate a list of REST endpoints defined in JAX-RS code
Background – JAX-RS
I’ve been working on a project that uses JAX-RS – the Java API for RESTful web services. If you don’t know JAX-RS, you write web services in Java using annotations to specify what REST endpoint a Java method implements.
For example, you can use @Path annotations on a class to define the root URI for methods in the class, and then use annotations like @GET, @Produces(MediaType.APPLICATION_JSON) and @Path on the individual class methods to define the endpoints that they implement.
The problem?
Reading from code to the web service is straightforward enough. By which I mean, if I’m looking at a Java method, it’s easy enough to look at it and know what endpoint it is implementing.
Going the other way can be a little trickier.
Once a project gets bigger, you can have REST endpoints spread around a large number of classes. And methods can inherit attributes from other classes than the one they’re in, through annotations like @Parent.
What if I’m using one of the project’s REST APIs, and want to look at the source for the method that’s handling it, whether to extend it or fix a bug? How can I remember which method in which class is responsible for the REST endpoint I’m using?
Using Reflection
Documentation is one way. As I develop the code, maintain a list of the mapping of Java methods to web services endpoints. And keep that up-to-date as I make any changes to the code.
But that’s very manual, and doesn’t seem very smart.
This got me thinking yesterday evening. I’d not used Java Reflection before, but thought it must be possible to work it out from the Java annotations in the same way that my JAX-RS provider must.
So I spent a bit of time trying it out and thought it might be useful to share what I came up with. It’s not terribly elegant or efficient. It’s the result of a few hours tinkering. But it shows the basic idea, and that seems useful enough to warrant sharing.
What I came up with
I’ve shared the source below, but in short, I’ve written a small stand-alone Java app that uses Reflection to search through my code for methods with annotations indicating a web services endpoint.
Once it finds them all, it creates a document with the basic information about them – as a sortable, searchable HTML table, using the DataTables jQuery plugin.
(If you’ve not used DataTables before, it’s simple and very cool. It lets you turn a static HTML table into something that supports filtering and sorting without needing anything server-side. I’m creating a table modelled on one of their examples. You should give it a try.)
What can I do with it
This gives me a complete list of all the endpoints in my project, together with the Java class and method name that implements them.
Creating the list in an HTML table using jQuery and DataTables gave me a few extra tricks:
It’s sortable – I can click on a column heading to sort by URI, or by the REST method, or by Java class, etc. …
It’s searchable – typing a query into the search box above the table will filter the contents of the table.
For example, I can type in the URI that I’m interested in, and everything vanishes apart from the relevant endpoints.
Or I can type in DELETE to see just the DELETE endpoints I’ve implemented.
Or, because I’ve included stuff like payload and the query and path parameters in the table, I can type in something like sortOptions to filter the table to show the endpoints that support a sortOptions parameter – to show me which of my web services APIs support searching.
You get the idea.
Not sure how useful it’ll turn out to be, but it was an interesting thing to play with.
The source
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.lang.annotation.Annotation;
import java.lang.reflect.Method;
import java.net.URL;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
import javax.ws.rs.DELETE;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Request;
import org.apache.wink.common.annotations.Parent;
/**
* Explores the Java classes in a given package, looking for annotations
* indicating REST endpoints. These are written to an HTML table, documenting
* basic information about all the known endpoints.
*
* @author Dale Lane (dale.lane@gmail.com)
*/
public class RESTEndpointsDocumenter {
public static void main(String[] args){
RESTEndpointsDocumenter red = new RESTEndpointsDocumenter();
try {
// the root package where Java classes implementing web services
// endpoints can be found - the place to start the search from
String packagename = "uk.co.dalelane.myproject.rest";
// the file location where the HTML table listing endpoints
// information will be written
// this should be in a directory that already exists, and contains
// the unzipped contents of
// http://dalelane.co.uk/files/120114-datatables-assets.zip
String destinationHtmlPath = "/path/to/my/doc/folder/index.html";
List<Endpoint> endpoints = red.findRESTEndpoints(packagename);
File endpointsDoc = red.outputEndpointsTable(endpoints, destinationHtmlPath);
System.out.println("Complete. Written to " + endpointsDoc.getAbsolutePath());
}
catch (IOException e) {
e.printStackTrace();
}
catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
/**
* Writes the provided REST endpoints to an HTML file.
*/
public File outputEndpointsTable(List<Endpoint> endpoints, String htmlpath) throws IOException {
File docfile = new File(htmlpath);
checkHtmlAssetFiles(docfile.getAbsoluteFile().getParentFile());
FileWriter fstream = new FileWriter(docfile);
BufferedWriter out = new BufferedWriter(fstream);
out.write("<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">" + NEWLINE);
out.write("<html>");
out.write("<head>");
out.write("<style type="text/css">" + NEWLINE);
out.write("@import "demo_page.css";" + NEWLINE);
out.write("@import "header.ccss";" + NEWLINE);
out.write("@import "demo_table.css";" + NEWLINE);
out.write("</style>" + NEWLINE);
out.write("<script type="text/javascript" charset="utf-8" src="jquery.js"></script>" + NEWLINE);
out.write("<script type="text/javascript" charset="utf-8" src="jquery.dataTables.js"></script>" + NEWLINE);
out.write("<script type="text/javascript" charset="utf-8" src="FixedColumns.js"></script>" + NEWLINE);
out.write("<script type="text/javascript" charset="utf-8" src="RowGroupingWithFixedColumn.js"></script>" + NEWLINE);
out.write("</head>" + NEWLINE);
out.write("<body id="dt_example">" + NEWLINE);
out.write("<table cellpadding="0" cellspacing="0" border="0" class="display" id="endpoints">" + NEWLINE);
out.write("<thead><tr><th>URI</th><th>REST</th><th>java class</th><th>method</th><th>parameters</th></tr></thead>" + NEWLINE);
out.write("<tbody>" + NEWLINE);
for (Endpoint endpoint : endpoints) {
switch(endpoint.method){
case GET: out.write("<tr class='gradeA'>"); break;
case PUT: out.write("<tr class='gradeC'>"); break;
case POST: out.write("<tr class='gradeU'>"); break;
case DELETE: out.write("<tr class='gradeX'>"); break;
default: out.write("<tr>");
}
out.write("<td>" + endpoint.uri + "</td>");
out.write("<td>" + endpoint.method + "</td>");
out.write("<td>" + endpoint.javaClass + "</td>");
out.write("<td>" + endpoint.javaMethodName + "</td>");
out.write("<td>");
for (EndpointParameter parameter : endpoint.pathParameters) {
out.write("path {" + parameter.name + "} (" + parameter.javaType + ")<br/>");
}
for (EndpointParameter parameter : endpoint.queryParameters) {
out.write("query: " + parameter.name + " (" + parameter.javaType + ") ");
if (parameter.defaultValue != null){
out.write("default = "" + parameter.defaultValue + """);
}
out.write("<br/>");
}
for (EndpointParameter parameter : endpoint.payloadParameters) {
out.write("payload : " + parameter.javaType + "<br/>");
}
out.write("</td>");
out.write("</tr>" + NEWLINE);
}
out.write("</tbody>");
out.write("</table>");
out.write("</body></html>");
out.close();
fstream.close();
return docfile;
}
/**
* Verifies that the JS and CSS files required by the HTML table are present.
*/
private void checkHtmlAssetFiles(File directory) throws FileNotFoundException {
if (directory.exists() == false){
throw new FileNotFoundException(directory.getAbsolutePath());
}
File rowgroupingjs = new File(directory, "RowGroupingWithFixedColumn.js");
if (rowgroupingjs.exists() == false){
throw new FileNotFoundException(rowgroupingjs.getAbsolutePath());
}
File fixedcolumnsplugin = new File(directory, "FixedColumns.js");
if (fixedcolumnsplugin.exists() == false){
throw new FileNotFoundException(fixedcolumnsplugin.getAbsolutePath());
}
File pagecss = new File(directory, "demo_page.css");
if (pagecss.exists() == false){
throw new FileNotFoundException(pagecss.getAbsolutePath());
}
File tablecss = new File(directory, "demo_table.css");
if (tablecss.exists() == false){
throw new FileNotFoundException(tablecss.getAbsolutePath());
}
File headercss = new File(directory, "header.ccss");
if (headercss.exists() == false){
throw new FileNotFoundException(headercss.getAbsolutePath());
}
File datatablesjs = new File(directory, "jquery.dataTables.js");
if (datatablesjs.exists() == false){
throw new FileNotFoundException(datatablesjs.getAbsolutePath());
}
File jqueryjs = new File(directory, "jquery.js");
if (jqueryjs.exists() == false){
throw new FileNotFoundException(jqueryjs.getAbsolutePath());
}
}
/**
* Returns REST endpoints defined in Java classes in the specified package.
*/
@SuppressWarnings("rawtypes")
public List<Endpoint> findRESTEndpoints(String basepackage) throws IOException, ClassNotFoundException {
List<Endpoint> endpoints = new ArrayList<Endpoint>();
List<Class> classes = getClasses(basepackage);
for (Class<?> clazz : classes) {
Annotation annotation = clazz.getAnnotation(Path.class);
if (annotation != null) {
String basePath = getRESTEndpointPath(clazz);
Method[] methods = clazz.getMethods();
for (Method method : methods) {
if (method.isAnnotationPresent(GET.class)){
endpoints.add(createEndpoint(method, MethodEnum.GET, clazz, basePath));
}
else if (method.isAnnotationPresent(PUT.class)){
endpoints.add(createEndpoint(method, MethodEnum.PUT, clazz, basePath));
}
else if (method.isAnnotationPresent(POST.class)){
endpoints.add(createEndpoint(method, MethodEnum.POST, clazz, basePath));
}
else if (method.isAnnotationPresent(DELETE.class)){
endpoints.add(createEndpoint(method, MethodEnum.DELETE, clazz, basePath));
}
}
}
}
return endpoints;
}
/**
* Create an endpoint object to represent the REST endpoint defined in the
* specified Java method.
*/
private Endpoint createEndpoint(Method javaMethod, MethodEnum restMethod, Class<?> clazz, String classUri){
Endpoint newEndpoint = new Endpoint();
newEndpoint.method = restMethod;
newEndpoint.javaMethodName = javaMethod.getName();
newEndpoint.javaClass = clazz.getName();
Path path = javaMethod.getAnnotation(Path.class);
if (path != null){
newEndpoint.uri = classUri + path.value();
}
else {
newEndpoint.uri = classUri;
}
discoverParameters(javaMethod, newEndpoint);
return newEndpoint;
}
/**
* Get the parameters for the specified endpoint from the provided java method.
*/
@SuppressWarnings("rawtypes")
private void discoverParameters(Method method, Endpoint endpoint){
Annotation[][] annotations = method.getParameterAnnotations();
Class[] parameterTypes = method.getParameterTypes();
for (int i=0; i < parameterTypes.length; i++){
Class parameter = parameterTypes[i];
// ignore parameters used to access context
if ((parameter == Request.class) ||
(parameter == javax.servlet.http.HttpServletResponse.class) ||
(parameter == javax.servlet.http.HttpServletRequest.class)){
continue;
}
EndpointParameter nextParameter = new EndpointParameter();
nextParameter.javaType = parameter.getName();
Annotation[] parameterAnnotations = annotations[i];
for (Annotation annotation : parameterAnnotations) {
if (annotation instanceof PathParam){
nextParameter.parameterType = ParameterType.PATH;
PathParam pathparam = (PathParam)annotation;
nextParameter.name = pathparam.value();
}
else if (annotation instanceof QueryParam) {
nextParameter.parameterType = ParameterType.QUERY;
QueryParam queryparam = (QueryParam)annotation;
nextParameter.name = queryparam.value();
}
else if (annotation instanceof DefaultValue) {
DefaultValue defaultvalue = (DefaultValue)annotation;
nextParameter.defaultValue = defaultvalue.value();
}
}
switch (nextParameter.parameterType){
case PATH:
endpoint.pathParameters.add(nextParameter);
break;
case QUERY:
endpoint.queryParameters.add(nextParameter);
break;
case PAYLOAD:
endpoint.payloadParameters.add(nextParameter);
break;
}
}
}
/**
* Get the REST endpoint path for the specified class. This involves
* (recursively) looking for @Parent annotations and getting the path for
* that class before appending the location in the @Path annotation.
*/
private String getRESTEndpointPath(Class<?> clazz){
String path = "";
while (clazz != null){
Annotation annotation = clazz.getAnnotation(Path.class);
if (annotation != null){
path = ((Path)annotation).value() + path;
}
Annotation parent = clazz.getAnnotation(Parent.class);
if (parent != null){
clazz = ((Parent)parent).value();
}
else {
clazz = null;
}
}
if (path.endsWith("/") == false){
path = path + "/";
}
return path;
}
/**
* Returns all of the classes in the specified package (including sub-packages).
*/
@SuppressWarnings("rawtypes")
private List<Class> getClasses(String pkg) throws IOException, ClassNotFoundException {
ClassLoader classloader = Thread.currentThread().getContextClassLoader();
// turn package into the folder equivalent
String path = pkg.replace('.', '/');
Enumeration<URL> resources = classloader.getResources(path);
List<File> dirs = new ArrayList<File>();
while (resources.hasMoreElements()) {
URL resource = resources.nextElement();
dirs.add(new File(resource.getFile()));
}
ArrayList<Class> classes = new ArrayList<Class>();
for (File directory : dirs) {
classes.addAll(getClasses(directory, pkg));
}
return classes;
}
/**
* Returns a list of all the classes from the package in the specified
* directory. Calls itself recursively until no more directories are found.
*/
@SuppressWarnings("rawtypes")
private List<Class> getClasses(File dir, String pkg) throws ClassNotFoundException {
List<Class> classes = new ArrayList<Class>();
if (!dir.exists()) {
return classes;
}
File[] files = dir.listFiles();
for (File file : files) {
if (file.isDirectory()) {
classes.addAll(getClasses(file, pkg + "." + file.getName()));
} else if (file.getName().endsWith(".class")) {
classes.add(Class.forName(pkg + '.' + file.getName().substring(0, file.getName().length() - 6)));
}
}
return classes;
}
//
// used to store the collection of attributes for a web services endpoint
//
static final String NEWLINE = System.getProperty("line.separator");
enum MethodEnum { PUT, POST, GET, DELETE }
enum ParameterType { QUERY, PATH, PAYLOAD }
public class Endpoint {
String uri;
MethodEnum method;
String javaClass;
String javaMethodName;
List<EndpointParameter> queryParameters = new ArrayList<RESTEndpointsDocumenter.EndpointParameter>();
List<EndpointParameter> pathParameters = new ArrayList<RESTEndpointsDocumenter.EndpointParameter>();
List<EndpointParameter> payloadParameters = new ArrayList<RESTEndpointsDocumenter.EndpointParameter>();
}
public class EndpointParameter {
ParameterType parameterType = ParameterType.PAYLOAD;
String javaType;
String defaultValue;
String name;
}
}
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
January 04 2012
December 16 2011
The Conversational Internet
Imagine using the Internet as a blind person.
As an occasional web-developer, I had some awareness of the importance of accessibility for the web, but to be honest it was pretty superficial. You just add ALT tags to your images, make sure you can tab between all the controls on the page, and a screen-reader will sort out the rest, right?
I went to an event in London a couple of weeks ago, where the reality was brought home to me.
Screen-readers are not as intelligent or as helpful as I’d assumed. They just read out everything on the page.
Imagine a typical modern web app… for example, facebook. Start reading everything on the page, from the top left of the page, and carry on until you reach the bottom right. Imagine what that might be like.
The best analogy I can think of is to try and picture the worst possible automated phone menu experience. The sort of one where they read you a long list of almost-unintelligible options: “for blah-blah-blah, press 1, for blather-blather-blather, press 2, for something-or-other, press 3 …. for something-else-vague, press 9 …”
None of the options seem like an exact match for the task that you have in mind, and by the time you’ve got to the end, you can’t remember whether the option that sounded sort of vaguely similar was option 3 or option 4…
Imagine that for a web page. Apparently, a screen-reader can take three or four minutes to read out the contents of a typical web page today. Can you imagine an automated phone system that spent four minutes listing your options, then expected you to try and choose which one you wanted?
That’s the experience that many blind people face when trying to use modern web apps that we take for granted.
ALT tags are all well and good, but making a web page accessible isn’t the same as making it usable.
So… as geeks with a passion for technology and an interest in making the web useful to all, what can we do?
The Conversational Internet
That was the question posed by the RLSB – a society for the blind – at an event they hosted in London a couple of weeks ago.
The demo that they showed was a video of someone trying to perform a task on the web using a screen reader. Never mind the futurology concept bit towards the end for the moment – the important bit is in the first 01:30, with the demo of the problem.
They had a clear perspective of the problem, and a general vision of what could help.
They described it as “The Conversational Internet”. The question was whether it could be possible for blind users to interact with the web at a higher, much more task-oriented level.
Instead of being read out an endless list of links and fields on a page as options, why can’t a user just say what it is that they want to do? Clearly inspired by the promise in the marketing for the iPhone 4S’s Siri interface, the vision is kinda compelling.
I go to facebook, but no longer have to try and navigate to the text entry box for making a status update by tabbing through every entry field in turn until I get to one that sounds like it’s the right one. Instead I just say that I want to update my status.
Or I go to a website for my local leisure centre, and no longer have to listen to a list of every type of service and activity they provide. Instead I just ask what activities are available in my area at the weekend.
Is this possible? Can we do this?
Some ideas
This was the discussion that RLSB hosted – with representatives from tech companies like Google, Samsung, Cisco and IBM, consumer-facing businesses like the Post Office and RBS, academics from Universities like Queen Mary and Brighton, charities for the blind like RLSB and Vision Charity, and others.
My notes from the evening aren’t very complete, and I was remiss at noting who said what, so apologies to everyone that I will now fail to attribute, or just entirely misrepresent.
This was a very brief meeting. It was an opportunity to introduce the problem to us, and give us a short chance to bounce around some initial reactions. So these ideas were not fully-formed or thought through.
Many of ideas discussed seemed, to me, to fit into one of three general approaches:
Automated natural language analysis of web pages
Can automated natural language analysis interpret both a user’s request, and the contents of a web page, in order to perform the requested task without needing the user to choose from a list of options? Think of how IBM Watson demonstrated the potential of interpreting questions in, and identifying answers from, natural unstructured text – but on a much smaller and more focused scale.
Custom task support using underlying APIs
What about ignoring the web page altogether and using the underlying APIs that many web apps make available nowadays to build a conversational interface? Think of how the iPhone’s Siri has been programmed to provide a voice interface to the core phone apps like messaging and reminders, by being programmed to use the same underlying APIs that the UI apps use. Think of the sort of thing that zypr are doing.
Web developers providing support for a conversational interface
What about using more semantic markup on web pages so that screen-readers could be more intelligent?
Or, what about using the existing VoiceXML standard, and making it a standard alternate interface to web pages? We already have an accepted norm that web pages can define alternate representations in the metadata – such as specifying the location of an RSS or ATOM feed. So we already have an existing way to specify the location of a VoiceXML service. And this is a long-standing interface for defining voice-based interactions.
What if web developers used VoiceXML to define an alternate voice-interaction method for tasks that their site supports? Then a screen-reader could see when this was available, and use this instead of reading the contents of the page.
There were a lot of pros and cons discussed for all approaches (Remind me to hide when cameras appear, next time!).
The more automated approaches require the least rework by web developers, and so are less dependent on large-scale support to succeed. However, they are likely very technically complex. The less automated approaches seemed to be more technologically manageable, but would rely on enough web developers supporting the approach in order to be viable.
A number of hybrid approaches were discussed that could potentially take the best elements of each. For example, an automated approach could be prototyped to be good enough to work with a select number of web apps. If it can work enough to demonstrate the concept for a dozen most-used web apps, maybe this demonstration could help drive the enthusiasm and support necessary to encourage broader adoption?
What now?
RLSB are looking to pull together a group to properly look into the area. What is already possible? What has already been done but could be brought together or even just better signposted to users? What technologies are emerging in the next few years that we could start trying out now?
I’ve tried to explain what I understood from the meeting I attended, but I’m not writing this on behalf of IBM or RLSB’s group. I’m writing this as an individual who thinks this is an exciting and ambitious idea with potential applications that go far beyond accessibility for the blind, and I’m writing this to find out what you think.
If you want to know what they really think, find the RLSB at www.rlsb.org.uk/2011/conversation and on twitter at @RLSBcharity.
If you think you can help, let them know!
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
November 15 2011
The problem with Jeopardy!
The I’ve-been-blogging-for-five-years-but-am-still-paranoid-about-work-related-posts-being-misrepresented disclaimer:
I do work for IBM as a developer on Watson but that role doesn’t extend to this blog. My responsibilities for Watson are limited to writing code… so any ramblings here are my personal views, and not necessarily representative of IBM’s positions, strategy or opinions.
The first time most people saw or heard of Watson was on “America’s favourite quiz show”, Jeopardy!
And that association seems to have stuck. For the moment, at least, Watson’s identity seems to bound up with Jeopardy’s. It’s “the computer system that won on Jeopardy!”
I’m not sure that’s entirely a good thing.
Jeopardy! was a great demonstration of Watson’s capabilities in a lot of ways. It showed the breakthroughs in interpreting natural language, the breakthroughs in coming up with evidence and rationale and identifying the level of confidence in it’s answer’s, and the power of machine learning systems.
But… it does make it easy to misunderstand the future potential of the technology.
The Jeopardy! challenge was about man vs machine.
It was about a machine trying to beat the humans.
A machine that had to work entirely by itself to come up with answers.
And if it couldn’t come up with an answer immediately, or when didn’t have enough confidence in the answer that it did come up with, it’d give up, stay silent and not buzz in.
None of that describes the way a future Watson could work. In my view, Watson should be about man and machine working together.
Watson could help to make it’s users smarter.
When a Watson system returns it’s answers, it could be showing it’s users the sources that it found that were relevant to the question – showing the passages from documents that the user might not have read or even heard of, helping to make them more informed.
It could show the reasoning for why it had it’s level of confidence in the answer, helping them to consider new perspectives.
And it works both ways – the users will make Watson smarter.
When a Watson system doesn’t have enough evidence to be confident in it’s top answer, it wouldn’t have to give up there. Every interaction wouldn’t have to be a single, stand-alone question. Watson could ask follow-up questions, working with the user to gather more information, more context and more evidence, until the level of confidence in the top answer improves.
Not only that, the machine learning in Watson would mean that every time a user provided a correct answer, Watson would get that little bit better and smarter at handling questions like that in the future.
Reciprocal. Collaborative.
These are not the words that Jeopardy! brings to mind. But it’s the vision of the future that I look forward to.
Watson doesn’t need to be about providing a definitive, take-it-or-leave-it answer without any additional information. It could be about helping decision makers to have the relevant stuff from terrabytes of text, right at their finger tips.
Watson could be a well-informed assistant for human experts.
More like a librarian who knows all of the text that has been written about your field of interest, and can work with you to find just what you need to answer the question that you have.
Does that vision come across in a man vs machine challenge?
IBM has uploaded a number of Watson-related videos to YouTube. Some of the comments that get posted on them are positive.
Others, however…
That isn’t the only one. There are loads like this.
Or…
To be fair, not all of them are entirely serious.
Okay… let’s be honest. I’m using comments from YouTube to back up an argument.
This is kinda cheating. There isn’t a point of view too crazy that you couldn’t find some commenter on YouTube that would back it up.
And I like I say, there are positive comments there, too, so I don’t want to blow this out of proportion. It’s not actually that bad.
But I do think introducing Watson to people in an adversarial, man vs machine format, has had some impact on the way it has been perceived. And there are plenty of more thoughtful, considered pieces in the mainstream press that talk about Watson being better than doctors, rather than how a Watson system could be a tool to make doctors more effective.
It’s not just the Internet crazies that are looking at this through a lens of “Watson was better than Ken and Brad on Jeopardy, and it beat them. Next it’s gonna be better than my doctor and beat them at medicine.” And in an era of outsourcing, there are people writing that they fear Watson as an attempt to save money by replacing humans with computers.
It’s been well reported that the computer on Star Trek was an inspiration for many of the original researchers working on the Watson project.
Characters on Star Trek had a computer that they worked with collaboratively, interacting with it using natural language. They would ask the computer questions, like “why is X happening?” or “what will happen if we do Y?”. It wasn’t a vision of a future in which computers were replacing people, or implying that people will be subservient to computers.
That’s the sort of vision we need to explain the potential of Watson.
Or, to pick a more contemporary example, maybe iPhone’s Siri is the best thing that could have happened for Watson. It puts the idea of question answering in people’s hands, and is getting people talking about the potential without worrying about it trying to beat us or replace us.
When we’re ready to show the world the first proof-of-concepts applying Watson to real-world problems, it’ll be interesting to see how it’s perceived, and whether the man-vs-machine image sticks.
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
November 06 2011
Why I go to barcamps
Last week, I went to BarCamp London. As always, it was a great weekend.
If you’ve not been to something like this before, the idea is that it’s a conference where each attendee contributes a session. There’s no real theme – just talk about something interesting.
You end up with an agenda made up of the interests, hobbies and skills of the random collection of people who managed to get a ticket. It makes for a more varied and eclectic agenda then you get from a traditionally organised conference.
Some of the talks that I went to included:
- A discussion on Secure Coding – thought-provoking
- An overview of CoffeeScript – a language that can be compiled into JavaScript – interesting as I’d not tried it before
- Dom’s experiences in starting an online sweet shop – very funny
- Simon and Nat’s experiences in turning lanyrd from a side project to a start-up – fascinating to hear about pretty much the exact opposite of what I did (going the corporate route!) since we left Uni
- A demo of and discussion about an open source project to create an open source media centre – rethinking TV is an interest of mine
- A Q&A with a couple of Wikipedians – inspiring – I love the idea of Wikipedia but haven’t really felt comfortable to get involved before
- One fan’s interpretation of the Zelda series of games, in an attempt to make sense of them all with a single coherent timeline – mental but fun
- A developer from the BBC talking about how their team does Agile – interesting to see how it compares with us
- “Two security geeks ranting” – I can’t do this one justice – inspired, and possibly my favourite session of the weekend
- A hands-on, code-heavy live demo of how to hack with the Kinect – very cool
- A dry-run of some stand-up material by @lallyd
- A demo of the new APIs and tools for creating Firefox extensions – looks a lot easier than the old ones!
- An overview of work being done for Surrey Police using mobile apps and social media – hadn’t heard of this – sounds promising
- A talk on how to do stuff anonymously on the web – in an era of social media being used to disrupt oppressive regimes and whistle-blow corporate malpractice, this is important
- One person’s experience in improving their productivity, inspired by the Kanban system – unusual
- A discussion about the importance of context when designing user interactions
- A presentation on the Freebase open repository for structured data and more generally about semantic web – I’ve used freebase before, so great to get some background
- A talk about the history of the Zeppelin – bizarrely fascinating
- A talk about the Clock of the Long Now – lunacy
To be fair, as a range of talks, a lot of them are quite geeky. But that says more about my choices than the range available. There were also talks on:
- A beginner’s guide to Absinthe – including tasting
- Basic Italian for techies
- “How to meet Polish female geeks?” – no idea what that was about, but the title got people talking
- How to make your own jam
- Sculpting using laser cutters
- How to bake a better cake – also including tasting!
- A crash course in how to do cryptic crosswords – I’d seen a bit of this talk at Over The Air – it was brilliant
And loads, loads more.
Why do I go to barcamps? Because it’s fascinating. It’s a way to get exposed to more new things, ideas and projects than you could normally cram into a weekend. And I love that it works because you get such a varied and interesting group of people into one place, and encourage them to share what they know.
If you’ve not been to a barcamp before, you should give it a go. And BarCamp London is one of the best.
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
October 02 2011
My first experience using BlueVia APIs
I wrote yesterday about a quick hack I did at Over The Air using the BlueVia API. I thought it was worth a quick post to show just how simple it was.
Read yesterday’s post for background to the idea behind the hack, but in essence, what I wanted was:
- monitor the location of my mobile phone
- send an SMS to a different mobile number when my phone goes into a predefined known area
BlueVia provides an API that let me doing this using network operator data. In other words, nothing needs to run on my phone itself as location data is obtained from where O2 thinks my phone is.
This means there is no battery-life impact on the phone for this monitoring.
It also means this will work with any phone – from iPhones and Androids to cheap feature phones.
The whole thing took me less than an hour and needed only 90 lines of Python.
This is how I did it.
1. Downloaded the zip file from BlueVia’s github page
2. Opened the Readme.html file in the examples/python/Sample Code folder
3. Installed the Python prereqs listed in the Readme.html file using easy_install
4. Created an app at bluevia.com requesting the Send SMS and the Location API
I got keys and IDs immediately.
5. The Readme.html contained enough sample code to show me how to do their OAuth process:
# overtheair-auth.py
import bluevia
consumer = 'XXXXXXXXXXXXXX'
secret = 'XXXXXXXXXXXXX'
o3 = bluevia.BlueViaOauth(consumer, secret)
urlobj = o3.fetch_request_token()
code = urlobj[0]
url = urlobj[1]
if code == 200:
access_token = raw_input('go to ' + url + ' then come back here and enter the verifier\n\nverifier: ')
o3.fetch_access_token(access_token)
o3.saveAccessToken("blueviaauthtok.pkl")
else:
print 'fail ' + code
6. The Readme.html file also contained enough sample code to show me how to get my mobile’s current location
# overtheair-location-attempt1.py
import bluevia
import pprint
l = bluevia.BlueViaLocation()
l.loadAccessToken("blueviaauthtok.pkl")
location = l.locateTerminal()
pprint.pprint(location)
7. That wasn’t very accurate.
It gave me a location somewhere in Slough, which was nowhere near me in Bletchley Park. That confused me for a minute until I remembered that O2 are in Slough.
Comparing the location I got out of the BlueVia API with the O2 UK Head Office…
Ah… I guessed it wasn’t returning me inaccurate location data. It was returning me canned data, which they’ve set to their head office location.
Which is when I remembered that in the BlueVia APIs talk I went to, they mentioned that the REST API included a sandbox endpoint for testing and development.
I took a quick peek at the source code of the Python bluevia library I downloaded from github, and spotted that the BlueViaLocation function had an optional sandbox parameter which defaulted to ‘sandbox’.
A quick tweak:
# overtheair-location-attempt2.py
import bluevia
import pprint
l = bluevia.BlueViaLocation(sandbox="")
l.loadAccessToken("blueviaauthtok.pkl")
location = l.locateTerminal()
pprint.pprint(location)
Bingo – I had the latitude/longitude of my location in Bletchley Park.
8. I needed to know how to see how far this location was from the predefined known area. A quick Google turned some public domain Python for doing this at johndcook.com – so big thanks to John D. Cook for saving me having to do any complicated maths.
9. Back to the Readme.html file, to get sample code for sending an SMS:
# overtheair-sendsms.py
import bluevia
import pprint
targetMobileNumber = "447654123456"
s = bluevia.BlueViaOutboundSms(sandbox="")
s.loadAccessToken("blueviaauthtok.pkl")
r = s.sendSMS([targetMobileNumber], "message to send")
pprint.pprint(s.deliveryStatus(r[1]))
10. I did a little tinkering to dynamically set a polling frequency based on my distance from the target
# 1 miles per hour = 0.44704 metres per second
# my max speed = 70 mph
# my max speed = (70 * 0.44704) metres per second
pollinterval = ((70 * 0.44704) / distanceInMetres)
# lower limit - don't poll more frequently than once a minute
if pollinterval < 60:
return 60
else:
return pollinterval
11. Putting it all together, I had this:
# overtheair.py
import bluevia
import pprint
import math
import time
# borrowed with thanks from http://www.johndcook.com/python_longitude_latitude.html
def distance_on_unit_sphere(lat1, long1, lat2, long2):
# Convert latitude and longitude to
# spherical coordinates in radians.
degrees_to_radians = math.pi/180.0
# phi = 90 - latitude
phi1 = (90.0 - lat1)*degrees_to_radians
phi2 = (90.0 - lat2)*degrees_to_radians
# theta = longitude
theta1 = long1*degrees_to_radians
theta2 = long2*degrees_to_radians
# Compute spherical distance from spherical coordinates.
cos = (math.sin(phi1)*math.sin(phi2)*math.cos(theta1 - theta2) +
math.cos(phi1)*math.cos(phi2))
arc = math.acos( cos )
# return in metres
return arc * 6373 * 1000
def getlocationtoken():
l = bluevia.BlueViaLocation(sandbox="")
l.loadAccessToken("blueviaauthtok.pkl")
return l
def getlocation(l):
loc = l.locateTerminal()
return loc
def getdistanceinmetres(loc, target):
currentlat = float(loc[1]['terminalLocation']['currentLocation']['coordinates']['latitude'])
currentlon = float(loc[1]['terminalLocation']['currentLocation']['coordinates']['longitude'])
return distance_on_unit_sphere(target['latitude'], target['longitude'], currentlat, currentlon)
def sendsms(place):
s = bluevia.BlueViaOutboundSms(sandbox="")
s.loadAccessToken("blueviaauthtok.pkl")
r = s.sendSMS(["447654123456"], "I'm near " + place['name'] + " now. Do you need me to get you anything?")
pprint.pprint(s.deliveryStatus(r[1]))
def choosesleeptime(metres):
# 1 miles per hour = 0.44704 metres per second
# my max speed = 70 mph
# my max speed = (70 * 0.44704) metres per second
quickesttraveltime = ((70 * 0.44704) / metres)
# fix lower limit - don't poll more frequently than once a minute
if quickesttraveltime < 60:
return 60
else:
return quickesttraveltime
loctok = getlocationtoken()
while True:
loc = getlocation(loctok)
closestPlace = None
places = [ { 'latitude' : 51.99684, 'longitude' : -0.7384, 'name' : 'Bletchley Park' },
{ 'latitude' : 50.96948, 'longitude' : -1.35246, 'name' : 'Sainsburys' },
{ 'latitude' : 50.97966, 'longitude' : -1.34928, 'name' : 'Tesco Express' } ]
for place in places:
place['distance'] = getdistanceinmetres(loc, place)
if closestPlace is None:
closestPlace = place
elif closestPlace['distance'] > place['distance']:
closestPlace = place
if closestPlace is None:
print "something went wrong. fail."
exit()
# network location wont be precise, so tolerate anything within 500 metres
goodenough = 500
if closestPlace['distance'] < goodenough:
sendsms(closestPlace)
print "finished - don't do this again for at least 12 hours"
time.sleep(12 * 60 * 60) # 12 hours
else:
seconds = choosesleeptime(closestPlace['distance'])
print str(closestPlace['distance']) + " from nearest place. waiting for " + str(seconds) + " seconds before checking again"
time.sleep(seconds)
12. It worked.
Okay, so it was a silly idea.
But the point is - it was amazingly easy to get this working. To do this by writing a native mobile app would have needed much more effort, and would have only worked on one platform.
The downside? You need to be on a mobile network that supports BlueVia - which in the UK means O2. I couldn't have done this if I was on Vodafone. Which restricts the usability a little.
But still. It is pretty cool.
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
October 01 2011
Over The Air 2011
I’m back from Over The Air 2011 – a mobile tech conference with an overnight hack-a-thon challenge.
This was the fourth year I’ve been to OTA, and I normally submit some random hack.
In fact, this is the first time I didn’t stay up all night, in a 15-hour non-stop coding splurge writing a massive, over-ambitious beast of a mobile app. I must be getting old… this time I wrote a couple of quick hacks, each of which under a couple of hundred lines long, and had eight hours sleep instead. Much more civilised.
This is what I managed to come up with:
hack 1: a crap husband helper
Sometimes I stop at the shop on the way home from work because I want something. Invariably, I don’t think to check with Amy if we need anything. I get home with a shopping bag, and get something along the lines of “Why didn’t you tell me you were going to the shops? We need nappies.” At which point, I feel like I should dutifully volunteer to go back out to the shop.
Going to a talk on the BlueVia API gave me an idea.
I wrote a bit of Python to run on my home server, that will keep an eye on where my phone is using the BlueVia Location API.
If I go near the shops, it will send a message to my wife using the BlueVia Send SMS API, to say “I’m near <insert shop name here>, do you need me to get you anything?”.
90 lines of Python is all it takes to make me look like a thoughtful and considerate husband. Huzzah.
The API is a polling one, so I try to poll sensibly – deciding when to poll based on how far I am from the shops, and how fast I can possibly travel. For example, if I’m over 70 miles from the shops, there is no point polling again within an hour, as I couldn’t get there quicker without going consistently faster than 70 mph.
And it is completely cross-platform – the BlueVia script runs on my server and is getting my location direct from my network operator, O2. Nothing is running on my phone, so it doesn’t matter what type of phone I have. And it sends SMS, which can be received by any phone.
(Okay, so this is pretty stupid. If you want to see a proper use of the Location API, check out locateupdate.co.uk, which keeps Google Latitude up to date using location information from the network operator, and without needing to constantly run Google Maps on your phone, or even have an Android phone.)
hack 2: a day in the life
My other hack was a Windows Phone 7 app. My first attempt at WP7 development! I even had to download the WP7 developer tools today… in fact, that was probably the trickiest part of the hack, given the wifi difficulties* at the event.
For one day, every hour it will ask you to take a photo to capture the moment – to get a photo of what you are doing.
(And it uses a live tile on the home screen to remind you if you still haven’t taken your photo this hour.)
It gives you a scrolling picture gallery to show a day in your life.
Keeping a photo of where you have breakfast, or you in your office, or where you go to lunch every day with your friends, etc. might seem pointless now.
But in 10 years time, how awesome will it be to look back and see a typical day in photos… with all of the everyday stuff that’s so usual that you never bother to keep a photo of it?
What else?
I went to some fantastic talks – presentations that taught me stuff, and presentations that got me thinking.
For example, I picked up a bunch of Android development tips from Nick and Richard. And I learnt about a project to build a structured, linked data catalogue of the contents of multiple UK archives, including the BBC’s, from Mo.
Plus I got to go on a guided tour of Bletchley Park and learn about the history behind machines like Enigma and Colossus.
I also gave a talk on debugging mobile web apps which seemed to be well received.
Plus I got a load of cool stickers, T-shirts and mugs.
Even Grace is now WP7-branded. What’s not to like?
* I’m not moaning about the wifi. Providing wifi for a very large number of people in an unfamiliar location (each of whom had several WiFi-enabled devices with them) is not trivial.
Feed footer idea nicked from 43 Folders using the FeedEntryHeader WordPress plugin.
September 26 2011
IBM European WebSphere Technical Conference - IBM Training - Global
trying to write my keynote for WSTC next month http://t.co/KSCUCOQC or… to be more accurate, staring at a blank page.Qubulus - Indoor Positioning Service Provider
RT @oldmanuk: Any Hursley peeps w/ Android phones played with Qubulus/Gecko to do an indoors GPS map of the site’s buildings? http://t.c ...RT @amylane http://t.co/pXf8Yk20 Unexpected bonus! :-) < looks like you broke her! ;-)
RT @amylane http://t.co/pXf8Yk20 Unexpected bonus! :-) < looks like you broke her! ;-)September 25 2011
Seth's Blog
@spwilkinson_uk I seem to share stuff from http://t.co/9nr9Kybv http://t.co/u8XseO7Y http://t.co/bmyyv3jy http://t.co/Y7zix59R most oftenDaring Fireball
@spwilkinson_uk I seem to share stuff from http://t.co/9nr9Kybv http://t.co/u8XseO7Y http://t.co/bmyyv3jy http://t.co/Y7zix59R most oftenMaybe Soup is currently being updated? I'll try again automatically in a few seconds...
