Thursday, April 17, 2008

Web hosted cron job schedulers

Google App Engine (GAE) currently comes without CRON scheduling support, while this is likely to change in the future I thought i would check out whats available in web-based scheduling services. These services basically make a scheduled HTTP request to your server.

Authentication to your server seems like a good idea, or at least your scripts should be hidden under very hard to guess URLs

-- MG

Review: Web hosted cron job schedulers :: chrisTHIS!: "I use DreamHost to host my blog and while it’s a great and fairly inexpensive service, there are some limitations. One is that you cannot run cron jobs without a shell account, which I do not have. A cron job is a frequently executed task, mainly used for running scripts on a frequent basis for website maintenance. While I already do this on my Windows server and I could use it for my needs here at chrisTHIS.com, I wanted to see if there was another way to schedule a cron job for the average Joe Blogger.

I set out looking for web-based cron services and found a few. Here are my reviews of each. None of these are affiliate links, nor am I associated with any of these services. I am providing my honest personal opinions. If you know of more, please let me know!"

OnlineCronServices.com: "Cron is a utility found on Linux/Unix servers used for scheduling script execution. Not all hosting providers offer access to this utility. However, webmasters and developers can still enjoy the benefits of scheduling by using an online cron service. Below is a list of cron services that are hosted remotely.

Name	Starting Price	Free Accounts	Lang.
Cron Service	4.95EUR/yr		NL
WebCron.org	n/a		FR
Web Service Scheduler	$11.88/yr		EN
WebbasedCron	$19.95/yr	trial	EN
CronService	$19.95/yr	trial	EN
Onlinecron	n/a		FR
Cronjob.de	2.99EUR/mo	trial	DE
cronjobs.org	n/a		EN
cronjobs.com	5EUR/mo		DE
HostedCron	$10/mo		EN
web-cron.org	30EUR/?		FR
Remote-cron.com	n/a		EN
MyCronJobs.com	?		EN
Cron-Jobs.com	1.02EUR/mo		DE
Webgrid Scheduler	n/a		EN
crontastic.com	?	for beta	EN
Onlinecronjobs.com	n/a		EN

S3 Firefox Organizer

S3 Firefox Organizer: "S3Fox Organizer for Amazon(S3Fox) v0.4: (NEW!) This firefox extension(browser plugin) provides an user friendly interface for Amazon's S3 (Simple Storage Service) . Its interface is very much similar to the FTP interface that lists local folders in the left panel and S3 buckets/files/folders in the right panel. Files/folders can be moved from the local computer to Amazon's storage space and vice versa. Follow the pre-requisites & steps described below to start using S3Fox Organizer.

New features in v0.4:

- Drag-n-drop support
- Ability to overwrite folders when uploaded
- Adhoc synchronization, meaning you can add a folder to 'Synchronize folders' list at the time of transfer.
- Ability to specify secure protocol
- Folder synchronization based on timestamps i.e. ability to synchronize only unchanged files"

Amazon Web Services Developer Connection : Firefox Extension for Amazon EC2

Amazon Web Services Developer Connection : Firefox Extension for Amazon EC2: "Firefox Extension for Amazon EC2

This is a Mozilla Firefox extension for interacting with Amazon EC2 (API version 2007-08-29). The source code also functions as an example of how to use the Amazon EC2 Query API from JavaScript."

A video that walks through the process of running an Amazon Machine Image (AMI) from Windows.

Amazon EC2 Getting Started Video

See the Getting Started Guide and the related technical documentation for a written version of these steps. Also, check out the attached transcript of the video, courtesy of CastingWords and Amazon Mechanical Turk.

Website Optimizer

Website Optimizer: "Increase your site’s conversion rate

Website Optimizer, Google’s free website testing and optimization tool, allows you to increase the value of your existing websites and traffic without spending a cent. Using Website Optimizer to test and optimize site content and design, you can quickly and easily increase revenue and ROI whether you’re new to marketing or an expert."

Google vs. Microsoft in Data Center Economics - Data Center Knowledge

Google vs. Microsoft in Data Center Economics - Data Center Knowledge: "Many state and local governments have passed tax incentives to attract a Google data center, hoping it will convert struggling regions into high-tech hubs. It's part of a trend in which more economic development agencies are pursuing data center projects. There's no question that favorable environments can create data center clusters, a trend seen in many technology corridors. But here's the secret: If you're focused on developing a data center cluster in your area, you should be chasing Microsoft, not Google."

Google App Engine Blog: Getting your data on, and off, of Google App Engine

Google App Engine Blog: Getting your data on, and off, of Google App Engine: "One upcoming feature of Google App Engine that's worth mentioning is large-scale data import and export. We know that many of you have large apps that already run on other platforms, and getting all of that data into the Datastore API is a challenge. And, conversely, we also know that a lot of you want to make sure it's easy to move off Google App Engine in the future, in case you need to switch to a different platform.

With Google App Engine, you own all the data in your app. As stated in our terms, you always have the right to get your data out of Google App Engine at any point. We wouldn't have it any other way."

Service Level Automation in the Datacenter: Google App Engine: How AppDrop Does and Does Not Affect Lock-in

Service Level Automation in the Datacenter: Google App Engine: How AppDrop Does and Does Not Affect Lock-in: "The cloud computing blogging world is abuzz with the news that Chris Anderson has created an interesting experiment in which he has created GAE-compliant hosting on Amazon EC2. This is an experiment that I've actually been looking at myself, but having a day job that is intensely busy right now, haven't had a chance to get to.

What did Chris do? In short, he got a working copy of the GAE SDK working on Amazon virtual machines with some modification. There are limitations:

* It will not scale (none of the Google 'secret sauce').
* It does not support email at this time (though the source code is available for anyone who wants to add it).
* It could go down at any time, either due to an EC2 outage (not very likely), or because whoever is paying for this doesn't want to foot the bill anymore (much more likely).

The implications of this are big, however. A couple of days ago, in an email exchange with Simon Wardley, I did a little analysis of what it would take to do a scalable version of this in open source. I don't have the depth of knowledge necessary to identify all of the relevant projects, but here's what I came up with:"

tecosystems � Clouds Rolling In: The Google App Engine Q&A

tecosystems � Clouds Rolling In: The Google App Engine Q&A: "“If Amazon/eBay/Google/Yahoo offer platforms - and they will, I believe.” - me

That was Saturday. By Monday, Google App Engine had launched with much fanfare. When I said cloudy days were ahead, I obviously believed that to be the case. I just didn’t know that meant Monday.

Some forecaster I am.

Anyway, you’re hearing it here last (seriously) as usual, but as the highest profile entrant into the cloud computing market since Salesforce.com, Google has earned itself a Q&A. Better late than never and all that."

What Exactly Is Google App Engine?

Google hides infrastructure from AppEngine users. AE programmers never (and, in fact, aren't allowed to) think about database scaling and configuration, load balancing , fail-over, etc. In theory, the complexity of writing a highly scalable app completely disappears.

With EC2, you still have to set-up load balancers, configure multiple replicated database servers, implement scalability hacks if things grow too fast (such as distributed caching of data via memcached), keep distros and apps up-to-date, etc. Bottom Line: EC2-based companies still require sys admins, AppEngine companies don't. That will certainly change as more companies begin offering EC2 server management services.

Google provides a non-relational datastore and that's the only datastore available (no traditional file system, no relational databases). With EC2, people generally use MySQL or Postgresql. Amazon offers a non-relational datastore called SimpleDB, but it's a bit *too* simple. For example, it does not support sorting of results sets. Huh? That makes it non-workable in my opinion. There's also an issue with using EC2 virtual machines for your database servers -- Amazon says that when a virtual machine crashes, all the data managed by it disappears, so virtual machine crash = hard drive crash.

With EC2, programmers can use any (non-Microsoft) language to develop their apps. AppEngine users must code in Python. Also, Google does not support sockets at this time. All cross-app communication must be done via HTTP.

At *this* moment in time, it would be difficult to move apps off of AppEngine. Doing that in EC2 is trivial. This, to me, is the biggest issue, as I believe it could make startups less-interesting from an acquisition perspective by anyone other than Google. This will most likely change as people develop compatibility layers. However, Google has yet to provide any information about how to migrate data from their datastore the best I can tell. If you have a substantial amount of data, you can't just write code to dump it because they will only let any request run for a short period before they terminate it.

Some people are complaining about Google having access to their source code. I don't see this as an issue. I'd rather have it be stored at Google than at some small hosting company.

One final nice little thing in AppEngine's favor: Websites that store less than 500MB of data and get roughly 5MM pageviews per month or less can use AppEngine for free. The downside is that Google has yet to say what they'll charge if apps go over that quota, but I have to believe that it will be reasonable. Right now, you're prevented from going above the free-level quotas.

YouTube - yael naim new soul clip

YouTube - yael naim new soul clip: "yael naim shows the first video clip of her new album"

YouTube - Missy Higgins - Where I Stood

YouTube - Maria Guttesen - Wide Open

An Introduction to ZooKeeper Video

March 7, 2008

http://developer.yahoo.com/blogs/hadoop/2008/03/intro-to-zookeeper-video.html

A few weeks ago, I had the chance to capture video of a presentation given by Benjamin Reed from Yahoo! Research. His presentation was an introduction to ZooKeeper, a highly available and reliable coordination system built by Yahoo! Research and released under the Apache License, Version 2.0.

Preparing to post the video, I asked Ben for a a summary of the motivations for building ZooKeeper. Here's what he had to say:

In 2006 we were building distributed applications that needed a master, aka coordinator, aka controller to manage the sub processes of the applications. It was a scenario that we had encountered before and something that we saw repeated over and over again inside and outside of Yahoo!.

For example, we have an application that consists of a bunch of processes. Each process needs be aware of other processes in the system. The processes need to know how requests are partitioned among the processes. They need to be aware of configuration changes and failures. Generally an application specific central control process manages these needs, but generally these control programs are specific to applications and thus represent a recurring development cost for each distributed application. Because each control program is rewritten it doesn't get the investment of development time to become truly robust, making it an unreliable single point of failure.

We developed ZooKeeper to be a generic coordination service that can be used in a variety of applications. The API consists of less than a dozen functions and mimics the familiar file system API. Because it is used by many applications we can spend time making robust and resilient to server failures. We also designed it to have good performance so that it can be used extensively by applications to do fine grained coordination.

We have found ZooKeeper to be applicable to many distributed applications inside of Yahoo! and expect it to be applicable to many more outside of Yahoo! For that reason we released it as open source under the Apache license. If you are writing a distributed application, ZooKeeper can help.

YouTube - Amy Winehouse=Rehab

YouTube - Duffy - Mercy - Official Music Video

YouTube - Gabriella Cilmi - Sweet About Me

New Blogger Tag Cloud / Label Cloud

New Blogger Tag Cloud / Label Cloud: "Here is the code and setup information to use the Label Cloud in New Blogger.

Wednesday, April 16, 2008

Global Domination � Yahoo! User Interface Blog

Global Domination � Yahoo! User Interface Blog: "Global Domination
June 1, 2006 at 11:27 am by Douglas Crockford | In Development |

JavaScript, as realized in browsers, is a load-and-go programming language. Programs are delivered to the execution site as text. This is a good fit for the web, which is at its root a text delivery system. The program text is eval‘d, which compiles it into an executable form and then immediately executes it. That execution can leave artifacts in the window’s global object.

The global object is the global namespace that holds the top level functions and global variables. Variables which are not explicitly defined are implied global variables, and their names are also kept in the global object. This was a convenience for the little scripts that Navigator 2 was expected to support. In the decade since NS2, those little scripts have grown into Ajax applications, and the true nature of the global object is revealed:"

Global Domination, Part Two � Yahoo! User Interface Blog

Global Domination, Part Two � Yahoo! User Interface Blog: "Global Domination, Part Two
April 16, 2008 at 1:29 pm by Douglas Crockford | In Development |

As I continue the practice of the craft of programming, I am always examining my practices. Can I improve the patterns that I use so that I can make my programs clearer, stronger, better? This is particularly important when working with a language like JavaScript which has a bias that favors patterns that are confusing, weak, and worse.

One of the worst features of JavaScript is its reliance on global variables.
This can be mitigated with global abatement and the module pattern, which can significantly reduce the number of global variables that we need to declare.

But when we must declare a global variable, how best should we do that?
JavaScript provides three ways of declaring a global variable, and they all
have problems. Which is least worst?"