vmware vcloud

virtualization leader vmware has officially entered into the cloud computing space with vcloud. they've been promising this for over a year, and it has finally happened. the vcloud api allows for enterprises to deploy an internal cloud. this cloud, of course, is supposed to be based upon a software stack of vmware technologies. it does not seem possible to swap out vmware components for equivalent solutions from other providers. so, while this would enable vmware customers features such as elastic computing, i would argue that vendor lock-in excludes it from being an actual cloud.

webgl followup

a blogger has discovered webgl code in webkit, which is the rendering library for apple safari and google chrome.

cloud computing steals from microsoft

sam ramji, microsoft's open source guru, has announced that he is leaving microsoft for a cloud computing startup.  there's no indication yet on which startup it is, except that it's in silicon valley, which excludes companies with vested cloud interests like hp, amazon, redhat and cisco, as well as startups like eucalyptus and rightscale as they are not located in silicon valley.  perhaps cloudera?  the announcement

monitoring vm instances using ganglia

there was an excellent email (the author's name is matt massie) on the hadoop listserv on how to setup ganglia on an instance running within amazon.  the directions are not specific to amazon's aws, however, and is generalizable to any vm instance.  i've copied the email below.
---

Ganglia doesn't need to be patched to work.  The patches are for Hadoop if
you are running ganglia 3.1.x (because of a breaking change in the ganglia
message format from 3.0.x to 3.1.x).

Since you are running fedora, you should be able to bring ganglia up by
using the ganglia RPMs which are available in the fedora repo.

Try the following commands on each node in the cluster you want to monitor

# yum install ganglia-gmond
# service gmond start

You will need to open TCP/UDP port 8649 as well to allow ganglia
communication (see your iptables configuration).  You can verify that gmond
is working by connecting to TCP port 8649

$ telnet localhost 8649

You should see an XML description of the state of the cluster/node.  Let me
know when you are this far in the installation and I'll help you through the
next steps.

---
and some additional info from that same thread

Hadoop comes with a number of scripts to configure EC2 instances, see
"HADOOP_HOME/src/contrib/ec2/

bin"

If you take a look at "src/contrib/ec2/bin/image/hadoop-init" you will see
that it sets up Ganglia.

amazon's virtual private cloud

until now, one of the biggest strengths of amazon's cloud computing model is that it is geared towards small businesses and startups. yet simultaneously, the same model is also one of its biggest weaknesses, due to the lack of enterprise support. this has encouraged a host of competitors to fill in that void. today amazon enters that space as it announces that new functionality to provide an enterprise driven model of support for cloud computing, termed "virtual private cloud".

initially, i thought such an offering would bring gloom to competitors such as eucalyptus. speaking with a member of that group, however, indicates otherwise. they are excited, for two reasons. first, it validates their business model (not that they needed amazon's validation anyway). second, it encourages enterprises to try out their software, since eucalyptus can be run on software you already own. and because the api is the same to both clouds, such enterprises can contract additional hardware on demand. good news for eucalyptus indeed.

google voice & sms

another exciting piece of news today.  you can now forward sms texts sent to your google voice account to an email address.  even better, replying to that text in the email will send an sms back to the sending.  how cool is that?  communications in the cloud!  ;)   the official announcement.

eucalyptus enterprise edition (eee)

eucalyptus today has announced the first release for the enterprise.  the main feature that eee adds on top of the open source version is the inclusion of vmware's virtualization technology, thereby enabling enterprises already invested in vmware to not have to migrate to xen or kvm.  this will go will with the recent announcement by amazon for virtual private clouds, to be discussed in an upcoming post.  the eucalyptus press release is copied below for archival purposes.

---

Eucalyptus Systems Launches the Eucalyptus Enterprise Edition Which Includes Support for VMware Virtualization Technologies

Eucalyptus’ First Commercial Offering Enables Enterprises to Transform Multiple Data Center Virtualized Environments into a Powerful On-Premise Cloud
SANTA BARBARA, Calif. – Sept. 9, 2009 – Eucalyptus Systems, Inc., creator of the leading open source private cloud platform, today announced the company’s first commercial offering: the Eucalyptus Enterprise Edition (EEE). EEE enables customers to implement an on-premise Eucalyptus cloud with VMware^®’s industry-leading virtualization technologies, including vSphere^®, ESX^TM and ESXi^TM.
EEE is the only on-premise cloud computing solution available today for vSphere customers, providing a robust, affordable cloud computing solution that leverages their investment in VMware technologies. EEE also supports other hypervisors typically found in a data center, such as Xen and KVM, providing customers the flexibility to configure cloud solutions that best meet their infrastructure needs.
“Eucalyptus Systems’ mission has been to support the open source Eucalyptus on-premise cloud platform while also delivering solutions for large-scale enterprise deployments, and we are proud to launch the Eucalyptus Enterprise Edition as our first commercial offering for the enterprise,” said Dr. Rich Wolski, Eucalyptus Systems co-founder, CTO and former director of the Eucalyptus research project at the University of California, Santa Barbara (UCSB). “EEE represents the first step toward broader Eucalyptus-enabled cloud interoperability that leverages multiple virtualization environments and technologies.”
EEE is built on Eucalyptus -- an open source software infrastructure for implementing on-premise cloud computing using an organization’s own information technology (IT) infrastructure, without modification, special-purpose hardware or reconfiguration. Eucalyptus turns data center resources such as machines, networks, and storage systems into a cloud that is controlled and customized by local IT. Eucalyptus is the only cloud architecture to support the same application programming interfaces (APIs) as public clouds, and today Eucalyptus is fully compatible with the Amazon Web Services cloud infrastructure.
For EEE, Eucalyptus leverages vSphere, ESXi, and ESX virtualization technologies to provide an on-premise cloud in the data center. EEE also includes an image converter that helps users develop VMware-enabled Eucalyptus applications that are compatible with Amazon EC2. Moreover, Eucalyptus supports popular open source hypervisors such as KVM and Xen, enabling EEE customers to choose the most appropriate software stack for each cloud application while maintaining a single cloud API that is Amazon compatible.
"As enterprises and governments increasingly seek to leverage the benefits of cloud computing in on premise infrastructure, existing datacenters are poised for transformation," said Stephen O'Grady, Principal Analyst with RedMonk. "With the release of its Enterprise Edition, Eucalyptus is providing customers with the ability to blend their VMware, KVM and Xen assets into a single cloud fabric while ensuring compatibility with existing public cloud options."

legal documents for startups

cloud computing is the next hottest thing, and i'm sure that there will be many startups around this technology.  one of the areas for a startup to address is legal affairs.  businessinsider is providing some boilerplate legal documents for startups to use as a reference.

delta cloud

this past week redhat announced delta cloud.  it is supposed to be a single api that connects to multiple clouds.  the idea has good intentions, but i wonder if it will become like drmaa.  drmaa was supposed to be a single api for grid engines, but never really took off, because each grid engine had its own idiosyncracies, and being a single api, drmaa needed to be the lowest common denominator of all of them, and thus was not very useful.  besides, who really used multiple grid engines anyway?

only time will tell what will happen to the delta cloud.

upcoming cloud talk, 9 sep 2009

clicker.com is hosting a talk to the los angeles cloud computing meetup group next thursday, 9 september, 7:15pm. details of the talk are copied below.
---

presentation and live demo of 3Tera's AppLogic cloud computing platform

Bert Armijo (blog.3tera.com) will be present and give a live demo of the 3Tera's AppLogic cloud computing platform - its features and real use cases based customer deployments.

Description From AppLogic:
AppLogic cloud computing platform allows users to combine any number of virtual machines into hierarchical structures such as web services, clustered data bases, and whole applications supporting the most popular data center operating systems - Linux, Solaris and Windows. Two levels of composition are supported at this time, assembled appliances and applications. Assembled appliances are placed into the AppLogic catalog to operate as a class definition from which instances can be instantiated on-demand for user applications.
At the application level via a drag-and-drop intuitive GUI applications are created using the AppLogic infrastructure editor. The composition provides a standardized set of commands for starting/stopping/backing-up/scaling/migrating/parameterizing/instantiating applications. This uniform set of capabilities enables applications that are complete and totally independent from the infrastructure they run on, including storage, networking, resource budgets, and parameterization. The resulting applications are created "in the cloud" at runtime and are completely portable between physical data centers without modification.

hadoop on mac os x?

i have been trying to setup an hadoop cluster on some macs, but have run into the problem that apple has not yet released a version of java6, which hadoop requires, to leopard (os x 10.5.*). more specifically, apple has only released java6 for 64bit intel processors (i.e. those with core 2 monikers). it seems that there may be hope after all, as someone reports that snow leopard (10.6.*) has 32 bit java6.



looks like i may be upgrading my systems soon, though it won't do much good for my ppc-based machines.

upcoming cloud talk, 3 sep 2009

uuasc (unix users association of southern california) is hosting a talk about how to build custom images for amazon's ec2.  details (copied from the webpage) are below.
---

Next Meeting, Thursday Sept. 3rd , 7pm-9pm
Location: Sun Microsystems, El Segundo

Building Custom Linux Images for Amazon EC2

Eric Hammond

IMPORTANT: Please be prompt, as we plan to start at 7 pm SHARP !! The building is locked at 7 pm and anyone who arrives after that time will require a Sun employee to escort them to the meeting. It is also highly probable that those who arrive after 7:30 may not be able to attend the meeting.
This month we will stick our heads back into the clouds, specifically Amazon's Elastic Compute Cloud EC2, which uses Xen based virtualization to launch web facing servers on-demand.
At the September meeting of UUASC-LA, Eric Hammond, who hosted the EC2 BoF and the EC2 Linux for Beginners lab at SCALE7x, will expand on his recent presentation at OSCON 2009. The talk covers customizing Amazon Machine Images (AMIs) to create an Ubuntu or Debian machine image to your specs with a process you can modify and reuse as your needs change.
This talk expects that audience members are comfortable running and using a Linux system and have at least a basic familiarity with what Amazon EC2 is. Attendees who have run an instance on EC2 and have an understanding of shell scripting will get the most out of this presentation. Experts in virtual machine building will still pick up pointers for automating the process for EC2.
Eric Hammond is an Internet Startup Technologist who has been involved in the architecture, development, and leadership of a number of successful early stage startups including Citysearch.com, Stamps.com, Rent.com, ThisNext.com, and CampusExplorer.com.

Post meeting activities

Even if you eat pizza, you will still have room for dessert, so dont forget our optional gathering at Coco's restaurant in the Manhattan Beach mall, corner of Sepulveda and Marine Ave, for dinner and chatting. (Drive south of the meeting building down Sepulveda, turn left on Marine, and take an immediate left into the shopping area). The chocolate cream pie is a particular point of interest ;-)

---

Directions to Sun Microsystems, El Segundo

A big thank-you to Justin Roth of Sun, for providing us with our meeting place!

222 N. Sepulveda Blvd, Suite 1800, El Segundo, CA 90245
10th floor, to the right of the elevators.
Use the elevators on the west side of the building

This is one of the huge white "Pacific Towers" buildings. The one with
"Oracle" on the top, funnily enough...

From the 105 freeway:
  Take the Sepulveda (aka hwy 1, aka airport) exit, but go SOUTH.
  It's about a kilometre down Sepulveda, on the left.
  Take a left just BEFORE the building, on Grand avenue, and turn
  into "Visitor Parking"

From the 405 freeway:
  Take the exit just south of the 105 (El Segundo Blvd)
  Go west for 2+ miles on El Segundo Blvd.

  Turn right onto Sepulveda Boulevard. Stay in the right lane and take
  another right onto Grand Avenue. Pass the building on the corner
  completely and turn into the second driveway on the right were it says
  "Visitor Parking."

Parking:
Parking in the adjacent structure appears to have cycled back to
non-free, for now.

Here's a map for ya!

amazon goes after the enterprise cloud

one of the major concerns about cloud computing for enterprises is data security.  it's thus a well accepted fact that amazon's cloud offerings would never be taken seriously by enterprise corporations, especially those subject to the yokes of sarbanes-oxley.  companies like eucalyptus systems, which offers software for enterprises to build out their own internal private clouds have benefitted from this requirement.  that doesn't mean that amazon can't try to win those potential customers over, however.  they've announced the ability for enterprises to create virtual private clouds (vpc) within ec2.  i have not read too deeply about the specifics of this offering, but it does not seem like there are any guarantees about how the data stored by any enterprise customers can be segrated from the rest of the cloud, and still be afforded the same quality of service provided to non-vpc clients.  the aws blog post for the announcement is here.

oozie, a workflow management system for hadoop

yahoo is working on oozie, a workflow management system to layer on top of hadoop. there is not much details on it thus far. sources and some discussion of the system can be found at the jira ticket. alejandro abdelnur, the main developer at yahoo, also gave a talk (slides) about it at the hadoop summit 09 in santa clara on 10 august.

hadoop world 2009

cloudera is hosting hadoop world 2009 in nyc on 2 october.  they are definitly spending a lot of time and energy building the community.

pyside, nokia's python bindings for qt

earlier this year nokia, which bought out trolltech last year, announced that they would release the qt library under lgpl (press release). this was great news for any company that wanted to sell a commercial c++ application with a qt-enabled frontend without release all of their sources. my understanding with companies is that it's not they are not willing to pay for use of the source code; its that the per-developer licensing model which qt employed was simply too high for most companies. unfortunately, riverbank computing, which develops pyqt, the python bindings to qt, did not feel the same way, and continues to license pyqt under the old model.

pyside is nokia's effort at bringing python bindings to qt. it looks like they are using boost python to accomplish the task, making the task that much more daunting. pyside currently does not support python 3.0, windows, or mac os x, thereby minimizing its usefulness for many use cases, and defeats the cross-platform-ness of qt. i understand that it satisfies nokia's corporate needs however, as their products are linux based.

it would be a share for pyside to not expand its coverage. i suspect that nokia will put in that effort, albeit at a slow crawl. i also wonder whether nokia considered the option to buy out riverbank computing, the way they did for trolltech. perhaps they are introducing pyside as a way to disarm riverbank before any such negotiations. time will tell what happens.

doug cutting to join cloudera

cloudera is very hot these days. not only did they get $11m of funding within 3 months, in a very challenging vc environment, they have now recruited doug cutting, co-founder of the apache nutch, lucene, hadoop projects. mike cafarella, the other co-founder, is already in cahoots with cloudera. the press release is copied below for archival purposes.
---

Doug Cutting joins Cloudera

Back in October, I promised to keep marketing and sales out of this blog. We wanted to concentrate on technical topics and to choose signal over noise. Mostly, that’s meant that I let other people do the writing.
I’m breaking that habit today so that I can announce — with great pleasure! — that Doug Cutting, co-founder of the Apache Hadoop project and creator of Nutch and Lucene, has agreed to join Cloudera beginning on September 1, 2009. Doug’s contributions to Hadoop over the years have been considerable. With Yahoo!’s backing, he split the data-parallel processing engine out of the Nutch crawler to create the Hadoop project. He’s remained an active contributor and commentator, providing guidance and advice to the growing community of Hadoop users and developers.
Doug will join his project co-founder, Mike Cafarella, at Cloudera. Mike has a full-time appointment as a professor of Computer Science at the University of Michigan beginning in December. In the meantime, and part-time after he starts his academic work, Mike will be working as a consultant for us here.
In the near term, we expect no change to the specific project work Doug is doing. Cloudera is excited about Avro. We intend to continue to contribute to Hadoop and related projects, and we all expect that Doug will be a critical part of both our thinking and our activity there. Besides, of course, we’re very pleased to add such a capable and experienced systems engineer to our team.
We look forward to welcoming Doug on September 1!
—Mike Olson, CEO, Cloudera

webgl & webkit

khronos has announced a webgl initiative to bring hardware-accelerated 3d graphics (via javascript bindings) to the browser. a whole new world of possibilities once this happens. exciting! press release copied below.

---

Khronos Details WebGL Initiative to Bring Hardware-Accelerated 3D Graphics to the Internet

4th August, 2009 – New Orleans, SIGGRAPH 2009 – The Khronos™ Group, today announced more details on its new WebGL™ working group for enabling hardware-accelerated 3D graphics in Web pages without the need for browser plug-ins.  First announced at the Game Developers Conference in March of 2009, the WebGL working group includes many industry leaders such as AMD, Ericsson, Google, Mozilla, NVIDIA and Opera.  The WebGL working group is defining a JavaScript binding to OpenGL® ES 2.0 to enable rich 3D graphics within a browser on any platform supporting the OpenGL or OpenGL ES graphics standards.  The working group is developing the specification to provide content portability across diverse browsers and platforms, including the capability of portable, secure shader programs.  WebGL will be a royalty-free standard developed under the proven Khronos development process, with the target of a first public release in first half of 2010.  Khronos warmly welcomes any interested company to become a member and participate in the development of the WebGL specification.
The WebGL specification will leverage recent developments in Web technology including the Canvas element defined as part of the HTML 5 specification and the marked increases in JavaScript performance across all major browsers.  Accelerated OpenGL ES functionality that is directly accessible from JavaScript is expected to encourage a wide variety of 3D-enhanced Web applications including those using rich user interfaces for enhanced navigation and functionality - making the Web more enjoyable, productive and intuitive for end-users.
“The Web has already seen the wide proliferation of compelling 2D graphical applications, and we think 3D is the next step for Firefox. We look forward to a new class of 3D-enriched Web applications within Canvas, and for creative synergy between OpenGL developers and Web developers,” said Arun Ranganathan of Mozilla and chair of the WebGL working group.
“Google is committed to open web standards and is very excited to be part of the WebGL initiative,” said Matt Papakipos, engineering director at Google.  “We believe that WebGL is an important step toward making high-performance 3D possible in the browser.”
“The WebGL working group inside Khronos is a unique forum that is bringing together browser and silicon vendors to create a low-level, foundation API for 3D on the Web,” said Neil Trevett, president of the Khronos Group and vice president at NVIDIA.  “Khronos will be reaching out to the key Web standards groups and the wider community to ensure WebGL is an appropriate, dynamic and enabling piece of the Web ecosystem.”

cloud computing & the us government

the us government has released a rfq (request for quotation) for the cloud computing initiative. the document can be found here. reuven cohen, founder of enomaly, has a great writeup here.

eucalyptus upgrades

in addition to everything else, eucalyptus has been working with boto on building a new set of commandline tools that is agnostic of the underlying cloud controller, whether that is amazon's ec2 or their own offering.  the result of this effort is that with release 1.5.2, euca2ools is now available.  underneath, euca2ools uses the python based boto, but is meant (at least initially) as a seamless replacement of the commandline tools provided by amazon.  the eucalyptus public cloud has also been upgraded to 1.5.2.  rock on!