Computer Networks

Computer Networks Project 1a Prof. Jerry Breecher Networks - Project 1a

Networks - Project 1a

What You Will Do In This Project. The purpose of this project is to build programmatic interfaces to web applications. You have two tasks before you: • Develop a program that can “push a button” on a web page and can read data from that web page. • Develop a program that can interact with twitter.com and do social networking. Networks - Project 1a

A Web Application API Every web application has an implicit API (Application Programming Interface.) That application is taking in requests on a particular IP address and a particular port (most likely port 80). It typically takes in those requests from a web browser, but it could just as easily get those requests from a program you write. You will use such an interface in writing Part 1. The web application may also have an explicit API. In this case, the app is listening on some other port. The interface may support more or fewer features than does its web (port 80) interface. You will use such an interface in writing Part 2. There are of course many ways to learn. There are sequential learners, those who like to have knowledge laid out in an orderly fashion and they can easily gobble it up. The Wireshark labs we’ve been doing are of that nature. Another style of learning is called “global”. Put a piece of code in front of these kind of learners, and they ingest it – they look at it, see things they recognize, things they don’t recognize, and then go off on their own to figure it out. This lab is definitely in that second category. Networks - Project 1a

Part I – Mimicking a Web Browser In this first part, you will be writing code to talk with an application that has only a web interface. This means you will talk to port 80 on its host. When you push this physical button, the web page will say “button pushed = 0”. Please be very gentle with this button!!! When you push the “virtual button” on the web page, this LED will light up. The web page will also display LED status. Pins connect to inputs and outputs on PIC processor. Ethernet connection Power Input PIC Processor Networks - Project 1a

Part I – Mimicking a Web Browser In this first part, you will be writing code to talk with an application that has only a web interface. This means you will talk to port 80 on its host. Host Xavier Switch To talk to the PIC, Xavier needs to be on the same network as the PIC. This interface is configured to 192.168.0.5 The PIC card has its interface hardwired to 192.168.0.30 This interface on Xavier is configured to talk with the switch on the clark network. Xavier has two physical Ethernet interfaces. So to do this part of the project, you will need to log onto Xavier. Networks - Project 1a

Part I – Mimicking a Web Browser This is the output of running ifconfig on xavier. It shows the two hardware interfaces and the “local” loopback virtual interface. As you can see, one port is set to 140.232.101.129 and the other is 192.168.0.5. jbreecher@xavier:~$ /sbin/ifconfig eth0 Link encap:Ethernet HWaddr 00:12:3f:5a:ce:e9 inet addr:140.232.101.129 Bcast:140.232.101.255 Mask:255.255.255.0 inet6 addr: fe80::212:3fff:fe5a:cee9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth1 Link encap:Ethernet HWaddr 00:60:97:e0:7e:04 inet addr:192.168.0.5 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::260:97ff:fee0:7e04/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 Networks - Project 1a

When I access the web server on the PIC, here is what is returned. Here telnet makes the call, but you could use Firefox just as easily. Part I – Mimicking a Web Browser jbreecher@xavier:~$ telnet 192.168.0.30 80 Trying 192.168.0.30... Connected to 192.168.0.30. Escape character is '^]'. GET /index HTTP/1.1 HTTP/1.0 200 OK Content-type: HTTP/1.0 200 OK Content-type: <table border="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="AutoNumber1" cellpadding="2" height="328"> <tr> <td width="100%" colspan="2" height="69"> <font face="Arial"> <img border="0" src="Clark2b.gif" width="272" height="100"> </font> <img border="0" src="chip.jpg" width="130" height="130"> </font> </td> </tr> <tr> <td width="100%" bgcolor="#000000" colspan="2" height="16"> <p align="center"> </td> </tr> <tr> <td width="50%" height="19"> </td> <td width="50%" height="19"> </td> </tr> <tr> <td width="100%" colspan="2" height="186" > <font face="Arial"><font color="#000000"><strong> Welcome to the CS280 Web Server! </strong></font><b></a></b></font><p> <font size="2" color="#000000" face="Arial"> This server is running on a PIC 18F25J10 microcontroller that combines ethernet capability along with regular processing. </font></p> <p><font color="#000000" face="Arial" size="2"> The bottom portion of this page illustrates real-time control of the demo board. The Left section refreshes with board information every few seconds and the Right section issues commands to toggle the on-board LED. </font></p> <p><font color="#000000" face="Arial" size="2"> You can push the button on the processor board which will cause a change in the RESULTS section below. You can also click on the right-side button to toggle the on-board LED. </font></p> </td> </tr> <tr> <td width="100%" height="19" colspan="2" bgcolor="#FFFFFF"> </td> </tr> <tr> <td width="100%" height="19" colspan="2" bgcolor="#000000"> </td> </tr> <tr> <td width="100%" height="19" colspan="2" bgcolor="#FFFFFF"> </td> </tr> <tr> <td width="50%" height="19"><font face="Arial"> <iframe name="I1" src="Status.cgi" border="0" frameborder="0" marginwidth="1" marginheight="1" width="100%"> Your browser does not support inline frames or is currently configured not to display inline frames.</iframe> </font> </td> <td width="50%" height="19"><font face="Arial"> <iframe name="I2" src="Commands.cgi" border="0" frameborder="0" marginwidth="1" marginheight="1" width="100%"> Your browser does not support inline frames or is currently configured not to display inline frames.</iframe> </font> </td> </tr></table> Connection closed by foreign host. jbreecher@xavier:~$ Networks - Project 1a

Part I – Mimicking a Web Browser Here’s the web page served up by the PIC-MINI-WEB. Note this is the same thing as given on the previous slide, but now Firefox is interpreting the data. Compared to the last output, there are at least FOUR extra calls made to the PIC server in order to fill in everything on this page. Networks - Project 1a

Part I – Mimicking a Web Browser Get the status here: Is the light on? Is the button, located on the board, pushed? Programmatically, “push” this button in order to toggle the light. Networks - Project 1a

Part I – Mimicking a Web Browser • OK – so all this has been “setting the stage” for your project. The data you need to access is NOT on the telnet version of the page. But it IS on the Firefox version of the page. How did Firefox get it? • Your task is to read the PIC web page and to: • Get the LED status. • Determine if the switch on the board is pushed • To toggle the on off state of the LED on the board. • A reasonable way of going about this project is to use Wireshark to study what the browser is sending to the PIC server. From that you can determine what your program needs to send. • In Chapter 2 of the text, you see examples of Java code that does TCP work. I recommend you use that code, or equivalent code you find on the web to write this project – it’s far easier than writing it in C. • Major HINT – you only need to use GET commands here – no PUSH is necessary in order to activate the button – Wireshark will tell all. Networks - Project 1a

Part II – Talking to a Web Application Networks - Project 1a

Part II – Talking to a Web Application Your task in this part is to write a program that will interface with Twitter. Common ways used by the average citizen to do this interface are a web interface via Firefox for instance, or via a phone app. But you aren’t an average citizen, so you get to do it in a third way. By the way, I recommend you read the recent article in the Boston Globe about companies that are profiting by organizing the data found on Twitter. Given the program you are about to write, and a bit of entrepreneurship, you could be wealthy indeed. I have put the article in the Appendix. There are several tools that will make your life easy. The twitter API is described at http://apiwiki.twitter.com/. This contains all the information you need to write a program to interface with twitter. People have written java libraries that interface with Twitter. Your only task is then to interface with these libraries. I developed code by using “twitter4j”, but there are several tools available for this purpose. Networks - Project 1a

Part II – Talking to a Web Application • Whichever tool you use for this task, you should do the following: • Send a “what are you doing” message to your account. • Send a Direct Message tweet to breecher from your twitter account. • Search #jobs worcester and look for good jobs. • Do something else interesting Networks - Project 1a

Project Evaluation Sheet • Name: ___________________________________ • Code written that can access the microchip web server. • Program successfully obtains all necessary information from the server. • Program pushes “web button”. • Program screen scrapes the two status values and presents them in a coherent fashion. • You can send a “what are you doing” message to your Twitter account. • You can send a Direct Message tweet to breecher from your twitter account. • You can search #jobs Worcester and look for good jobs. • You can do something else interesting using your Twitter access program. Networks - Project 1a 15

Appendix Start-ups flock to Twitter, looking to capitalize on network’s growth By Scott Kirsner Globe Columnist / October 4, 2009 The central topic of conversation over the past few weeks, when two or more tech types gather in the same place, has been whether Twitter is in fact worth $1 billion, and what the company might eventually do to generate revenue. Though it is now three years old, the Internet messaging service still hasn’t revealed the particulars of its business plan. But that didn’t stop several investors, including Boston-based Spark Capital, from funneling another $100 million into Twitter last month, setting the San Francisco company’s value at $1 billion, at least on paper. In our neighborhood, more than half a dozen companies offer applications and services connected to Twitter - and some of them, unlike Twitter itself, are already starting to bring in actual money from customers. (What an old-school notion!) Call them Twitter remoras - companies attached to Twitter like remora fish attached to a shark. “Twitter, to me, is kind of like the phone company,’’ says Mike Langford, founder of Westborough-based Tweetworks LLC. “They’re building this great infrastructure. I think there are tons of opportunities for them to make money, and just like there is a lot of stuff built on the infrastructure of the phone system, I think you’ll see profitable companies built on Twitter.’’ Tweetworks aims to make conversations on Twitter a bit more organized. Its service, offered at no charge, helps organize Twitter messages, often called “tweets,’’ into clusters around a particular topic so they are easier to follow. Networks - Project 1a

Appendix Tweetworks users can also create their own groups of people interested in a topic. Tweetworks doesn’t generate revenue yet, but Langford expects that customers who want certain features, like maintaining private groups or getting statistics about group activity, may pay a monthly fee to use it, or that advertisers will want their online campaigns targeted to groups of Twitter users interested in, say, home improvement and interior design. The companies that seem to be gaining the most momentum are those that offer some sort of Twitter measurement service. Cambridge-based HubSpot Inc. offers a free service called Twitter Grader, which analyzes how well a Twitter account is performing in terms of attracting followers and supplying them with content that they’re likely to pass along (or “re-tweet’’) to other users. Twitter Grader is used tens of thousands of times every day, according to HubSpot cofounder Dharmesh Shah, and the free service helps attract paying customers to HubSpot, which sells online marketing software that starts at $250 a month. Crimson Hexagon, a Harvard spin-out, helps companies analyze what consumers are saying about their products across all sorts of online forums, like blogs, discussion groups, and Twitter. But “a fair number of our customers these days are interested only in Twitter,’’ says chief executive Candace Fleming. “We can summarize for them the opinions that are being expressed across thousands of tweets on a particular topic, like what people like or don’t like about Bing,’’ Microsoft’s new search engine. A typical client project that Crimson Hexagon works on, Fleming says, generates fees in the tens of thousands of dollars. The company raised about $1 million in new financing in August. Boston-based Traackr also sells analysis of what’s happening on Twitter to it clients. Traackr generates lists of influential users of Twitter and other social networking services; marketers may be interested in reaching out to these “influencers’’ to get them to test out a new website or sample a product. Clients pay Traackr based on the size of the list it generates, anywhere from $1,500 to $15,000 or more. Networks - Project 1a

Appendix GraphEdge is a service that creates a report to track trends among a Twitter user’s audience, like how quickly it is growing or shrinking. Created by Waldron Faulkner, GraphEdge is currently free, but Faulkner has plans to implement subscriptions soon for Twitter users with large numbers of followers. Oneforty is one of the newest local Twitter-related start-ups. It aims to provide a directory of the best (and worst) Twitter applications, like software that allows you to post messages from your mobile phone. Since some of these applications cost money, Oneforty’s business plan involves a small referral fee when it facilitates a transaction. Founder Laura Fitton has raised several hundred thousand dollars so far from angel investors. Among the Twitter remoras, there’s only a hint of worry that as Twitter grows and searches for sources of revenue, it may start duplicating what they’re doing, suddenly becoming a very big competitor. The crew at Thoughtbot, a Boston-based software development firm, built a site called PoliTweets last year, to collect what Twitter users were saying about the presidential race. Twitter mimicked that several months later with a site called Election.Twitter.com. “There’s obviously a risk when you’re building any of these apps that Twitter will do it for themselves,’’ says Dan Croak of Thoughtbot, which has built two other Twitter services, Pockets and Thunder Thimble. But most of the small companies don’t worry much about Twitter’s long-term viability. The service has simply gotten too popular too quickly, they say, for it to vanish. “I’m sure they must have some amazing plans for revenue,’’ says Fleming at Crimson Hexagon. Two more Twitter-linked entities locally are Spark Capital and Charles River Ventures in Waltham. Any eventual success or failure won’t have a big impact on Charles River, which has only about $250,000 invested in Twitter. (Twitter has raised $155 million in total.) Charles River participated in Twitter’s first round of funding in 2007, but wasn’t the lead investor - even back then, Charles River’s partners thought Twitter’s founders had too lofty a valuation in mind. Networks - Project 1a

Appendix But Twitter could be a reputation-making - or tarnishing - investment for Spark Capital, a relatively young firm. Partner Bijan Sabet sits on Twitter’s board and is an avid Twitter user. Spark took a hit earlier this year when another portfolio company, Verified Identity Pass Inc., which operated the Clear system for frequent fliers, abruptly shut down; Spark had led a $44 million funding round for the company just 10 months earlier. Whether Twitter matures into an independent, publicly traded company, or is acquired for an eye-popping sum by a tech giant like Google or Microsoft, a success will vindicate Spark - and tweak, just a little bit, their doubting rivals at Charles River. Langford, the founder of Tweetworks, says he has been trying to get an audience with Sabet at Spark to see if the firm might invest in his nascent business. But other local venture capital firms have been lukewarm. “You talk to some of the investors around here,’’ he says, “and they can’t understand the value of Twitter at all. They say, ‘We’re not sure how Twitter is going to make it, so how are you going to make it?’ ’’ Those are two big question marks in the Twitter ecosystem right now, though some of the Twitter remoras seem to be answering the revenue question more quickly than Twitter itself. Scott Kirsner can be reached at kirsner@pobox.com. Networks - Project 1a

Computer Networks