A COMMAND PROMPT TALK
This presentation is the property of its rightful owner.
Sponsored Links
1 / 85

A COMMAND PROMPT TALK PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on
  • Presentation posted in: General

A COMMAND PROMPT TALK. Node.js and postgres. An Introductory Glance With Live Code, Later. 2010 AURYNN SHAW, COMMAND PROMPT INC. Hi!. What is Node.js. Fast Pure event-driven server-side javascript programming environment. Ew, javascript. Built on Google’s V8

Download Presentation

A COMMAND PROMPT TALK

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A command prompt talk

A COMMAND PROMPT TALK


Node js and postgres

Node.js and postgres

  • An Introductory Glance

  • With Live Code, Later

2010 AURYNN SHAW, COMMAND PROMPT INC.


A command prompt talk

Hi!


What is node js

What is Node.js

  • Fast

  • Pure event-driven

  • server-side

  • javascript programming environment


Ew javascript

Ew, javascript

  • Built on Google’s V8

  • Runs on modern unix-likes (Linux, OSX, Freebsd, Cygwin)

  • Speaks network natively, and easily (Hello world is in http!)

  • Ends up Similar to Python, Perl, Ruby, ETc.


Did i mention fast

Did I mention fast?

  • Single thread can do many thousands of connections

  • standard library doesn’t block


Did i mention fast1

Did I mention fast?

  • Single thread can do many thousands of connections

  • standard library doesn’t block

  • Keeps the slow tasks from blocking

  • Keeps the quick tasks quick


Blocking

blocking?

# Standard synchronous processing

fh = open(“somefile”,”r”)

contents = fh.read()

# what if somefile is huge, 1GB+ ?

print contents


Every step waits

Every step waits


How is node different

How is node different?


Did i mention fast2

Did I mention fast?

  • Single thread can do many thousands of connections

  • standard library doesn’t block

  • Keeps the slow tasks from blocking

  • Keeps the quick tasks quick


Really good for anything i o

Really good for anything I/O


While slow tasks run node carries on

While slow tasks run, node carries on


Non blocking

Non-blocking!

fs.open(“somefile”, function (fd) {

// When the file descriptor gets returned.

var buffer = null;

fs.read(fd, buffer, length, position, function (e, bytes) {

// bytes read are here.

// buffer will be not-null.

});

});


As always there s caveats to this

As always, there’s caveats to this


Some caveats

Some caveats

  • node is entirely single-threaded

  • Event-driven programming is *really* different


A maze of callbacks

A maze of callbacks

fs.open(“/tmp/hello”, function (fd) {

var buffer = null;

fs.read(fd, buffer, length, position, function (e, bytes) {

// bytes read are here.

// buffer will be not-null.

});

});


Some caveats1

Some caveats

  • node is entirely single-threaded

  • Event-driven programming is *really* different

  • Javascript’s prototype model is also really different

  • The syntax has warts


Build a bridge out of her

Build a bridge out of her?

/*

This allows you to kind of inherit properly.

*/

var o = function () {};

var p = function () {

/* Because ‘this’ changes */

var self = this;

o.call(this);

};

p.prototype = o;

p.prototype.constructor = p;


So why is this important

So why is this important?


Why is it important

Why is it important?

  • Javascript owns the frontend


Why is it important1

Why is it important?

  • Javascript owns the frontend

  • More jquery and javascript programmers every day


Why is it important2

Why is it important?

  • Javascript owns the frontend

  • More jquery and javascript programmers every day

  • Javascript is fast


Faster every day

Faster every day


And now we can use postgres

And now, we can use postgres!


When i got here

When I got here...

  • postgres.js worked.


When i got here1

When I got here...

  • postgres.js worked.

  • Used the simple query protocol


When i got here2

When I got here...

  • postgres.js worked.

  • Used the simple query protocol

  • Used SQL escape to prevent injection attacks.


Whyyyy

whyyyy

SOURCE: http://www.flickr.com/photos/striatic/2192192956/


But it did work

But it did work!


And so i wanted to make it better

And so I wanted to make it BETTER


So i had to ask

So, I had to ask,


How do i not allow this

How do I not allow this?


A command prompt talk

Well...


To prepare a query

to prepare a query

  • postgres.js Didn’t use libpq.


A command prompt talk

SOURCE: http://www.flickr.com/photos/tanaka/2319325408/


To prepare a query1

to prepare a query

  • postgres.js Didn’t use libpq.

  • Speaks pure protocol


To prepare a query2

to prepare a query

  • postgres.js Didn’t use libpq.

  • Speaks pure protocol

  • ...So I had to learn the protocol.


Wax on wax off

Wax on, wax off

MONTAGE!


Sadly reality was a lot less exciting

Sadly, reality was a lot less exciting


Complexity abounds

Complexity Abounds

  • Simple protocol sends one message to pg

  • Easy to queue, internally.


A command prompt talk

REALLY SIMPLE MESSAGE QUEUE

QUERY

QUERY

QUERY

QUERY

QUERY

QUERY

RESULTS


Really though

Really, though

  • Simple protocol sends one message to PG

  • Easy to queue, internally.

  • Prepared statements require at least 5

  • Parse, describe, execute, bind, flush


A command prompt talk

THIS TYPE OF QUEUE DOESN’T WORK

SYNC

EXECUTE

BIND

DESCRIBE

PARSE

QUERY

QUERY


And while all this is happening

And while all this is happening

  • You cannot block the main thread

  • And postgres expects a synchronous request/response

  • But it does give responses...


And while all this is happening1

And while all this is happening

  • You cannot block the main thread


And while all this is happening2

And while all this is happening

  • You cannot block the main thread

  • And postgres expects a synchronous request/response


And while all this is happening3

And while all this is happening

  • You cannot block the main thread

  • And postgres expects a synchronous request/response

  • But it does give responses...


So we buffer message sets

So, we buffer message sets


A command prompt talk

KIND OF LIKE THIS

SIMPLE

PREPARED

GROUP

SIMPLE

SIMPLE

PREPARED

GROUP

SIMPLE

SIMPLE


So we never block waiting for the db

So we never block waiting for the DB


The application keeps responding

the application keeps responding


But that s all a bit on the technobabbly side

But that’s all a bit on the technobabbly side

SO


Let s look code

Let’s look codE!


Using postgres js

Using postgres.js

/* Node setup */

var pg = require(“lib/postgres-pure.js”),

sys = require(“sys”);

var db = new pg.connect("pgsql://test:[email protected]:5432/template1");

/* We now have a working DB handle.

Or, we have an error. Either way.

*/


Using postgres js1

Using postgres.js

/* We now have a working DB handle. */

/* From here, we do: */

db.query(“select 1::int;”, function (err, rs) {

// Wait, a function?

console.log(sys.inspect(rs));

});


Why a function

Why a function?


What s going on here

What’s going on here?

/* We now have a working DB handle. */

/* From here, we do: */

db.query(“select 1::int;”, function (err, rs) {

// Wait, a function?

console.log(sys.inspect(rs));

});


What you get

What you get

1

3

// A little while later

2


It s happening asynchronously

It’s happening asynchronously!

  • The function argument is what’s going to happen when the DB comes back

  • db.query returns immediately

  • and your app keeps on going!


This isn t the cool bit

this isn’t the cool bit


Prepared queries

Prepared queries

/* Prepared queries work the same way */

db.prepare(“select ?::int as foobar;”, function (sth) {

sth.execute(1, function (err, rs) {

console.log(sys.inspect(rs));

});

sth.execute(2, function (err, rs) {

console.log(sys.inspect(rs));

});

});


Prepared queries1

Prepared queries

/* which will return */

[ { foobar: 1 } ]

[ { foobar: 2 } ]


Or this

Or this!

/*

This becomes a prepared statement behind the scenes

*/

db.query(“select ?::int as foobar;”,

1, // A single argument

function (err, rs) {

console.log(sys.inspect(rs));

});


It would return

It would return

[ { foobar: 1 } ]


Sub ideal aspects

Sub-ideal Aspects


Bad bad bad

Bad, bad, bad.

  • postgres.js is still very alpha.

  • Node API is in flux

  • Learning Node takes a while


Cool bits

Cool bits!


Things that are pleasing

Things that are pleasing

  • clean, perl DBI-based API.

  • Official support from Command prompt

  • Performance!


Vroom

Vroom

[ananke:~/src/postgres-js] aurynn> time node benchmark.js 40000

[ { count: 40001 } ]

node benchmark.js 40000 18.85s user 2.75s system 87% cpu 24.602 total

Just over 2122 queries/second.


Some stuff we can t wait to add

Some Stuff we can’t wait to add


Shiny things

Shiny things!

  • Notify support


Pg notifications

PG notifications

/* Event-driven frameworks are ideal for this, and

the notification setup can be handled behind the

scenes.

*/

db.on(“<NOTIFYNAME>”, function (payload) {

// Do something useful as a result of

// the notification from PG

});


Payload is also arbitrary data

Payload is also arbitrary data


Shiny things1

Shiny things!

  • Notify support

  • Stability


Shiny things2

Shiny things!

  • Notify support

  • Stability

  • Better error handling


Shiny things3

Shiny things!

  • Notify support

  • Stability

  • Better error handling

  • More test cases


Shiny things4

Shiny things!

  • Notify support

  • Stability

  • Better error handling

  • More test cases

  • Better type coercion


But the most important thing we d like

But the most important thing we’d like?


Users

Users.


You guys people who care

You guys. People who care.


So that s postgres js

So that’s postgres.js.


Any questions

Any questions?


Okay live code shout out ideas

Okay. LIVE CODE!Shout out ideas!


Https github com commandprompt postgres js

FORK IT, AND HELP!

UNSTABLE

https://github.com/commandprompt/postgres-js/

https://github.com/aurynn/postgres-js/

NOT QUITE AS UNSTABLE


  • Login