Quercus on Google App Engine
Ever since Google App Engine (GAE) supported Java, it has opened a slew of other languages that GAE indirectly and unofficially supports. PHP is one of them through Quercus, our 100% Java clean-room implementation of the PHP language.
PHP is unique among the other languages in that there are plenty of high-quality, killer applications (i.e. Wordpress, Mediawiki, and Drupal) that are free and readily available for it. Furthermore, there has been incessant calls for Google to support PHP from the very beginnings of GAE.
Unfortunately, GAE doesn’t support MySQL, or any other relational database for that matter. We’re limited to Google Datastore, which is pretty much a big hash table. So running existing PHP applications unmodified on GAE is currently out of the question because they are so dependent on the LAMP architecture.
But how hard is it to modify an existing PHP application to run on GAE? I set to gain insight on this interesting question by getting Wordpress running on GAE. Since Wordpress already runs on Resin with Quercus, all I needed to do was convert the MySQL queries to instead use the App Engine’s subset of JPA. In the end, it took about a week of uneventful work to get enough of Wordpress running so we could demo it at Google I/O.
Converting Wordpress to use JPA
The first step I undertook was to dump the database into JPA. To do so, I created a JPA entity class for each database table. The relationship from a database table to a JPA class is fairly straightforward. The table name maps to the fully qualified name of the class. Each field of the class represents a column in the table, with the field type following the column datatype as closely as possible. For example, the the wp_users table would map to something like the following JPA class:
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
/*
CREATE TABLE `wp_users` (
`ID` bigint(20) unsigned NOT NULL auto_increment,
`user_login` varchar(60) NOT NULL default ”,
`user_pass` varchar(64) NOT NULL default ”,
`user_nicename` varchar(50) NOT NULL default ”,
`user_email` varchar(100) NOT NULL default ”,
`user_url` varchar(100) NOT NULL default ”,
`user_registered` datetime NOT NULL default ‘0000-00-00 00:00:00′,
`user_activation_key` varchar(60) NOT NULL default ”,
`user_status` int(11) NOT NULL default ‘0′,
`display_name` varchar(250) NOT NULL default ”,
PRIMARY KEY (`ID`),
KEY `user_login_key` (`user_login`),
KEY `user_nicename` (`user_nicename`)
);
*/
@Entity
public class User
{
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long _ID;
private String _userLogin;
private String _userPass;
private String _userNicename;
private String _userEmail;
private String _userUrl;
private String _userRegistered;
private String _userActivationKey;
private Long _userStatus;
private String _displayName;
}
The varchar and int types map to Java String and Long respectively. I could have used Java Date for datetime but didn’t in the end because Wordpress does all the date/time processing on its end on strings anyway. For simplicity, I omitted the Java getters and setters.
I then created a script to create and persist each table row in JPA.
The next step and final step is to convert MySQL queries to JPA queries. This is the most time-consuming step because Wordpress didn’t try to abstract the database layer from the core code. In other words, large chunks of the code are practically doing direct MySQL queries. My job was made somewhat easier by the fact that JPA queries use nearly the same syntax as SQL queries. Take the following code that retrieves the comments for a particular post:
{
//SELECT * FROM $wpdb->comments WHERE comment_post_ID = %d AND comment_approved = ‘1′ ORDER BY comment_date
$query = $this->pm->createQuery(”SELECT FROM ” . GAE_COMMENT_CLASS . ‘ WHERE _commentPostID = :id AND _commentApproved = “1″ ORDER BY _commentDate’);
$query->setParameter(”id”, $post_id);
$results = $query->getResultList();
$array = array();
foreach ($results as $comment) {
$obj = new stdClass();
$obj->comment_ID = $comment->getCommentID();
$obj->comment_post_ID = $comment->getCommentPostID();
$obj->comment_author = $comment->getCommentAuthor();
$obj->comment_author_email = $comment->getCommentAuthorEmail();
$obj->comment_author_url = $comment->getCommentAuthorUrl();
$obj->comment_author_IP = $comment->getCommentAuthorIP();
$obj->comment_date = $comment->getCommentDate();
$obj->comment_date_gmt = $comment->getCommentDateGmt();
$obj->comment_content = $comment->getCommentContent();
$obj->comment_karma = $comment->getCommentKarma();
$obj->comment_approved = $comment->getCommentApproved();
$obj->comment_agent = $comment->getCommentAgent();
$obj->comment_type = $comment->getCommentType();
$obj->comment_parent = $comment->getCommentParent();
$obj->user_id = $comment->getUserId();
$array[] = $obj;
}
return $array;
}
As you can see, it was a one-to-one conversion for this particular query. The most crucial part was to make sure I was returning the correct type that Wordpress is expecting to get back. In this case, I needed to return an array of objects.
Wordpress does sparingly use more complex query features like joins, which of the course GAE doesn’t support (yet). However, it was fairly easy for me to replicate the behavior of joins on the application side with nested loops of two or more JPA queries. Joins are evil according to Google so I do not want to bore you with the details.
After rewriting about fifty queries, I had Wordpress running satisfactory enough to demo it at Google I/O. I can edit, create, and moderate comments and posts (though I can’t guarantee any capability beyond that :). The demo is running on the App Engine at http://wordpress-on-quercus.appspot.com/wordpress-2.7.1/. The source for the war is available here.
All in all, it was a fun experience learning the ins and outs of GAE. I wouldn’t say it was feasible to adapt an existing application to Google Datastore via JPA, but as a demo it served its purpose of showcasing quercus’ seamless integration of PHP and Java.
Looking forward, we may make it brain-dead to run existing PHP applications unmodified on GAE. We already have a file-based relational database that we use for our cache and logging. We could route MySQL queries to that JNDI database instead. Since GAE doesn’t allow file writes, we’ll just then store the file-based database in Google Datastore as one big blob of data. This will allow anyone to run unmodified LAMP applications on GAE with Quercus. Now that will be very cool!
Tags: google app engine, google datastore, jpa, mysql, php, quercus, wordpress
May 31st, 2009 at 8:31 am
It’s very interesting! But the link for the source is broken. Could you publish the source?
May 31st, 2009 at 12:13 pm
I fixed the link - please give it another try.
Thanks,
Emil
May 31st, 2009 at 12:26 pm
wow, nice demo !
about using the datastore for file storage, it would be nice if this is implemented in Quercus as a stream, so that something like below will be possible.
$data = file_get_contents(’datastore://some-thing-here’);
file_put_contents(’datastore://some-thing-here’, $data);
however, using ‘Google Datastore as one big blob of data’ for a file database (like sqLite does) does that not mean that it must retrieve & store the complete blob for every update?
btw: I have updated my SQL to JDO wrapper, see http://blog.herbert.groot.jebbink.nl/2009/05/rom-relational-object-mapping.html for details.
May 31st, 2009 at 1:53 pm
Yes, it would be possible to store files into the datastore using the standard php function calls.
When I said to store everything as one big blob of data, I was simplifying things. The Datastore limits entities to 1MB I think, so we would need to split the data into blocks of data.
May 31st, 2009 at 3:26 pm
Ok, sorry for a blog you must simplify, it is not a technical design
Lets dream: A VFS above the datastore and every cluster of it is an entry in the datastore. It uses memcache for caching. This VFS is mounted as / for PHP scripts. ( ./ and deeper is of course stil the war ) PHP scripts that use /tmp for example are then stil working. Oh, and also make it available as web-dav in Resin
May 31st, 2009 at 11:44 pm
Quercus on Google App Engine…
Ever since Google App Engine (GAE) supported Java, it has opened a slew of other languages that GAE indirectly and unofficially supports. PHP is one of them through Quercus, our 100% Java clean-room implementation of the PHP language….
July 19th, 2009 at 7:47 am
[...] run some PHP on GAE, what’s the first thing that comes to mind? Right… Try to run WordPress on GAE, of course [...]
July 19th, 2009 at 9:36 pm
Can I install it on Resin Open Source ?
I tried install it on Google App Engine (Resin 4 Open Source)
It gave me this error:
java.lang.NoClassDefFoundError: javax/inject/manager/Bean
Here is the log:
com.caucho.quercus.servlet.GoogleQuercusServlet service: java.lang.NoClassDefFoundError: javax/inject/manager/Bean
java.lang.NoClassDefFoundError: javax/inject/manager/Bean
at com.caucho.amber.manager.AmberPersistenceProvider.createEntityManagerFactory(AmberPersistenceProvider.java:63)
at javax.persistence.Persistence.createFactory(Persistence.java:172)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:112)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:66)
at com.caucho.gae.wordpress271.EMF.(EMF.java:9)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
…
July 20th, 2009 at 9:59 am
mikewen,
It should work fine with Quercus Open Source (in fact until recently, it only worked with Quercus Open Source). At this time it’s not possible to run Resin in GAE because they force the use of Jetty. Based on your stack trace, it looks like you’re trying to use our JPA implementation, which is also not supported for GAE. You’ll need to configure the GAE JPA implementation. Finally make sure that you include all the .jars included with Resin.
We’ll have more in depth instructions soon for Quercus on GAE.
Thanks for trying it out!
Emil
July 21st, 2009 at 9:12 am
Could you please change the link for the source?just don’t use GAE space for download.
if I downloaded it with a low speed,I couldn’t finish it at all.
July 23rd, 2009 at 7:37 am
Emil,
Thanks for your help.
I was able to run very simple php_info();
(Resin in GAE, I think should be Quercus Open Source)
The problem looks like JPA implementation not supported in GAE.
I will wait for your instructions.
(I know Python/PHP well, but know very little Jave )
Mike,
August 10th, 2009 at 8:53 pm
[...] the rest job is convert mysql to the Goole App Engine???s subset of JPA, for details, please visit Quercus on Google App Engine. Share and [...]
August 11th, 2009 at 8:48 pm
[...] for details, please visit Quercus on Google App Engine. [...]
August 16th, 2009 at 6:21 pm
You might want to try using jiql to automatically wrap the SQL queries.
August 17th, 2009 at 5:48 am
hi, i find this post very informative, i have downloaded the rar file from your source, could you please tell how to install the wordpress in GAE and i am new to GAE also, but i am technical person. can you help me in this matter
August 26th, 2009 at 6:46 am
Thanks a lot for your work.
I just add a build.xml file to enhance classes easily. Could you give me the admin password of the worpress blog ? (http://wordpress-on-quercus.appspot.com/wordpress-on-gae-quercus.zip)
I don’t anderstand how the datastore work and I can’t edit the admin password field …
September 2nd, 2009 at 9:17 am
It’s nice this implementation. Thank you very much for the contribution. However, I encountered this issue — GAE reported always some problems on “cpu quotas”, when I tried to “login”, or to execute “test.php”, “query.php”, etc. even run the most simple function “php_info()”. Anyone can help
To “june”, you can update the password filed to “c4ca4238a0b923820dcc509a6f75849b” (which means “1″ as password in MD5) by GAE’s data viewer. Then you can login by “admin/1″
October 11th, 2009 at 3:27 am
I have a question,which is my app engine could only support files smaller than 1MB
but http://wordpress-on-quercus.appspot.com/wordpress-on-gae-quercus.zip
is 9.3MB,much larger than it,could you kindly tell me how to upload files larger than 1MB?
October 31st, 2009 at 4:21 am
[...] ???GAE?????????wordpress???????????????????????????PHP???????????????PHP???Quercus??????????????????????????????Java???PHP???????????????????????????quercus?????????Google App Engine??????????????????Quercus on Google App Engine??????Quercus???GAE?????????????????????WP??????????????????????????????????????????????????????????????????????????????wp-db.php???????????????????????????JPA??????????????????????????????????????????????????????RSS??????????????????????????????????????? ???????????????jiql – ??????????????????????????????GAE?????????Mysql?????????????????????http://code.google.com/p/jiql/???jiql?????????GAE????????????MySQL??????????????????jiql??????????????????jiql???Quercus??? ?????????????????????jiql???Quercus?????????WP???PhpMyAdmin???????????????????????????????????????PHP?????????????????????????????????????????????????????????????????????jiql??????INSERT???????????????WP??????????????????????????????????????????10??????????????? ????????????WP2.8.5????????????????????????????????????????????????Java????????????java.net.Proxy????????????http.php????????????????????????????????????Socks?????????????????????????????????????????????????????????????????????????????????????????????require??????wp-settings.php:352????????????????????????????????? [...]
January 2nd, 2010 at 12:30 pm
First, thanks for great work!
I am trying to run this, first locally out of the box (with only two changes relating to appengine project name). I run the project like this:
./[..]appengine-java-sdk-1.3.0/bin/dev_appserver.sh .
and get the message “The server is running at http://localhost:8080/”
At http://localhost:8080/wordpress-2.7.1/wp-admin/install.php I got this error:
[..]wordpress/wordpress-2.7.1/wp-admin/includes/schema.php:22: Fatal Error: Call to undefined method GaeDb::has_cap()
I commented out lines 22 to 28 and now I reached a wordpress install screen and entered an email and blog name, but then I get more errors. I can get rid off one by changing a line in upgrade.php::1549 to $result=1;
However I think this error here is end of the line for me without major tinkering:
[..] wordpress-2.7.1/wp-admin/includes/upgrade.php:1140: Fatal Error: Call to undefined method GaeDb::get_col()
Is there a way I can get this to run?
March 6th, 2010 at 12:34 pm
how do i config the wp-config.php file?
DB_NAME
DB_USER
DB_PASSWORD
DB_HOST
what shall i fill in with？
and can u release a latest modified wp？
April 15th, 2010 at 6:28 am
[...] Converting Wordpress for Quercus on Google App Engine [...]
July 30th, 2010 at 7:19 pm
hello you guys, I set up a wordpress2.7.1 on google app engine, but when I log in, and click the admin site, I always get something like:
/base/data/home/apps/iamawordpressapp/7.343726665455507144/wordpress-2.7.1/wp-includes/user.php:364: Fatal Error: Call to undefined method GaeDb::prepare()
how to fix this?
February 16th, 2011 at 6:00 pm
Hi there! I’m converting some files that still not converted or presents some error, from this wordpress version, as I complete it, I’ll put the link here!
December 6th, 2012 at 6:54 am
[...] 3.5 years ago to this day since I last wrote about running Quercus on Google App Engine (GAE). The article detailed a crazy experiment to get a PHP application, Wordpress, running on GAE. I still remember [...]