20090829

hadoop on mac os x?

i have been trying to setup an hadoop cluster on some macs, but have run into the problem that apple has not yet released a version of java6, which hadoop requires, to leopard (os x 10.5.*). more specifically, apple has only released java6 for 64bit intel processors (i.e. those with core 2 monikers). it seems that there may be hope after all, as someone reports that snow leopard (10.6.*) has 32 bit java6.



looks like i may be upgrading my systems soon, though it won't do much good for my ppc-based machines.

20090828

upcoming cloud talk, 3 sep 2009

uuasc (unix users association of southern california) is hosting a talk about how to build custom images for amazon's ec2.  details (copied from the webpage) are below.
---


Next Meeting, Thursday Sept. 3rd , 7pm-9pm
Location: Sun Microsystems, El Segundo


Building Custom Linux Images for Amazon EC2

Eric Hammond

IMPORTANT: Please be prompt, as we plan to start at 7 pm SHARP !! The building is locked at 7 pm and anyone who arrives after that time will require a Sun employee to escort them to the meeting. It is also highly probable that those who arrive after 7:30 may not be able to attend the meeting.
This month we will stick our heads back into the clouds, specifically Amazon's Elastic Compute Cloud EC2, which uses Xen based virtualization to launch web facing servers on-demand.
At the September meeting of UUASC-LA, Eric Hammond, who hosted the EC2 BoF and the EC2 Linux for Beginners lab at SCALE7x, will expand on his recent presentation at OSCON 2009. The talk covers customizing Amazon Machine Images (AMIs) to create an Ubuntu or Debian machine image to your specs with a process you can modify and reuse as your needs change.
This talk expects that audience members are comfortable running and using a Linux system and have at least a basic familiarity with what Amazon EC2 is. Attendees who have run an instance on EC2 and have an understanding of shell scripting will get the most out of this presentation. Experts in virtual machine building will still pick up pointers for automating the process for EC2.
Eric Hammond is an Internet Startup Technologist who has been involved in the architecture, development, and leadership of a number of successful early stage startups including Citysearch.com, Stamps.com, Rent.com, ThisNext.com, and CampusExplorer.com.

Post meeting activities

Even if you eat pizza, you will still have room for dessert, so dont forget our optional gathering at Coco's restaurant in the Manhattan Beach mall, corner of Sepulveda and Marine Ave, for dinner and chatting. (Drive south of the meeting building down Sepulveda, turn left on Marine, and take an immediate left into the shopping area). The chocolate cream pie is a particular point of interest ;-)



Directions to Sun Microsystems, El Segundo

A big thank-you to Justin Roth of Sun, for providing us with our meeting place!
222 N. Sepulveda Blvd, Suite 1800, El Segundo, CA 90245
10th floor, to the right of the elevators.
Use the elevators on the west side of the building

This is one of the huge white "Pacific Towers" buildings. The one with
"Oracle" on the top, funnily enough...

From the 105 freeway:
  Take the Sepulveda (aka hwy 1, aka airport) exit, but go SOUTH.
  It's about a kilometre down Sepulveda, on the left.
  Take a left just BEFORE the building, on Grand avenue, and turn
  into "Visitor Parking"

From the 405 freeway:
  Take the exit just south of the 105 (El Segundo Blvd)
  Go west for 2+ miles on El Segundo Blvd.

  Turn right onto Sepulveda Boulevard. Stay in the right lane and take
  another right onto Grand Avenue. Pass the building on the corner
  completely and turn into the second driveway on the right were it says
  "Visitor Parking."

Parking:
Parking in the adjacent structure appears to have cycled back to
non-free, for now.

Here's a map for ya!

20090826

amazon goes after the enterprise cloud

one of the major concerns about cloud computing for enterprises is data security.  it's thus a well accepted fact that amazon's cloud offerings would never be taken seriously by enterprise corporations, especially those subject to the yokes of sarbanes-oxley.  companies like eucalyptus systems, which offers software for enterprises to build out their own internal private clouds have benefitted from this requirement.  that doesn't mean that amazon can't try to win those potential customers over, however.  they've announced the ability for enterprises to create virtual private clouds (vpc) within ec2.  i have not read too deeply about the specifics of this offering, but it does not seem like there are any guarantees about how the data stored by any enterprise customers can be segrated from the rest of the cloud, and still be afforded the same quality of service provided to non-vpc clients.  the aws blog post for the announcement is here.

20090823

oozie, a workflow management system for hadoop

yahoo is working on oozie, a workflow management system to layer on top of hadoop. there is not much details on it thus far. sources and some discussion of the system can be found at the jira ticket. alejandro abdelnur, the main developer at yahoo, also gave a talk (slides) about it at the hadoop summit 09 in santa clara on 10 august.

20090821

hadoop world 2009

cloudera is hosting hadoop world 2009 in nyc on 2 october.  they are definitly spending a lot of time and energy building the community.

20090820

pyside, nokia's python bindings for qt

earlier this year nokia, which bought out trolltech last year, announced that they would release the qt library under lgpl (press release). this was great news for any company that wanted to sell a commercial c++ application with a qt-enabled frontend without release all of their sources. my understanding with companies is that it's not they are not willing to pay for use of the source code; its that the per-developer licensing model which qt employed was simply too high for most companies. unfortunately, riverbank computing, which develops pyqt, the python bindings to qt, did not feel the same way, and continues to license pyqt under the old model.

pyside is nokia's effort at bringing python bindings to qt. it looks like they are using boost python to accomplish the task, making the task that much more daunting. pyside currently does not support python 3.0, windows, or mac os x, thereby minimizing its usefulness for many use cases, and defeats the cross-platform-ness of qt. i understand that it satisfies nokia's corporate needs however, as their products are linux based.

it would be a share for pyside to not expand its coverage. i suspect that nokia will put in that effort, albeit at a slow crawl. i also wonder whether nokia considered the option to buy out riverbank computing, the way they did for trolltech. perhaps they are introducing pyside as a way to disarm riverbank before any such negotiations. time will tell what happens.

20090811

doug cutting to join cloudera

cloudera is very hot these days. not only did they get $11m of funding within 3 months, in a very challenging vc environment, they have now recruited doug cutting, co-founder of the apache nutch, lucene, hadoop projects. mike cafarella, the other co-founder, is already in cahoots with cloudera. the press release is copied below for archival purposes.
---

Doug Cutting joins Cloudera

Back in October, I promised to keep marketing and sales out of this blog. We wanted to concentrate on technical topics and to choose signal over noise. Mostly, that’s meant that I let other people do the writing.
I’m breaking that habit today so that I can announce — with great pleasure! — that Doug Cutting, co-founder of the Apache Hadoop project and creator of Nutch and Lucene, has agreed to join Cloudera beginning on September 1, 2009. Doug’s contributions to Hadoop over the years have been considerable. With Yahoo!’s backing, he split the data-parallel processing engine out of the Nutch crawler to create the Hadoop project. He’s remained an active contributor and commentator, providing guidance and advice to the growing community of Hadoop users and developers.
Doug will join his project co-founder, Mike Cafarella, at Cloudera. Mike has a full-time appointment as a professor of Computer Science at the University of Michigan beginning in December. In the meantime, and part-time after he starts his academic work, Mike will be working as a consultant for us here.
In the near term, we expect no change to the specific project work Doug is doing. Cloudera is excited about Avro. We intend to continue to contribute to Hadoop and related projects, and we all expect that Doug will be a critical part of both our thinking and our activity there. Besides, of course, we’re very pleased to add such a capable and experienced systems engineer to our team.
We look forward to welcoming Doug on September 1!
—Mike Olson, CEO, Cloudera

20090805

webgl & webkit

khronos has announced a webgl initiative to bring hardware-accelerated 3d graphics (via javascript bindings) to the browser. a whole new world of possibilities once this happens. exciting! press release copied below.

---

Khronos Details WebGL Initiative to Bring Hardware-Accelerated 3D Graphics to the Internet


4th August, 2009 – New Orleans, SIGGRAPH 2009 – The Khronos™ Group, today announced more details on its new WebGL™ working group for enabling hardware-accelerated 3D graphics in Web pages without the need for browser plug-ins.  First announced at the Game Developers Conference in March of 2009, the WebGL working group includes many industry leaders such as AMD, Ericsson, Google, Mozilla, NVIDIA and Opera.  The WebGL working group is defining a JavaScript binding to OpenGL® ES 2.0 to enable rich 3D graphics within a browser on any platform supporting the OpenGL or OpenGL ES graphics standards.  The working group is developing the specification to provide content portability across diverse browsers and platforms, including the capability of portable, secure shader programs.  WebGL will be a royalty-free standard developed under the proven Khronos development process, with the target of a first public release in first half of 2010.  Khronos warmly welcomes any interested company to become a member and participate in the development of the WebGL specification.
The WebGL specification will leverage recent developments in Web technology including the Canvas element defined as part of the HTML 5 specification and the marked increases in JavaScript performance across all major browsers.  Accelerated OpenGL ES functionality that is directly accessible from JavaScript is expected to encourage a wide variety of 3D-enhanced Web applications including those using rich user interfaces for enhanced navigation and functionality - making the Web more enjoyable, productive and intuitive for end-users.
“The Web has already seen the wide proliferation of compelling 2D graphical applications, and we think 3D is the next step for Firefox. We look forward to a new class of 3D-enriched Web applications within Canvas, and for creative synergy between OpenGL developers and Web developers,” said Arun Ranganathan of Mozilla and chair of the WebGL working group.
“Google is committed to open web standards and is very excited to be part of the WebGL initiative,” said Matt Papakipos, engineering director at Google.  “We believe that WebGL is an important step toward making high-performance 3D possible in the browser.”
“The WebGL working group inside Khronos is a unique forum that is bringing together browser and silicon vendors to create a low-level, foundation API for 3D on the Web,” said Neil Trevett, president of the Khronos Group and vice president at NVIDIA.  “Khronos will be reaching out to the key Web standards groups and the wider community to ensure WebGL is an appropriate, dynamic and enabling piece of the Web ecosystem.”