Launching Resin and the Watchdog
The way that you start and stop Resin and the watchdog have changed a bit in the 4.0 branch. I realized as I was writing the Resin Refcard that there are probably a few options and concepts that many users don’t fully understand. In this post, I review the Resin watchdog architecture and show the syntax while highlighting recent changes. I’ll also describe some of the thoughts we have for the future of the watchdog and solicit your ideas and input.
Watchdog Architecture
Starting in Resin 3.1, we changed the way that Resin is started. With the Watchdog architecture, we moved from a single JVM running Resin to a two JVM model - one with the actual Resin process and one for the Watchdog. The Watchdog is responsible for starting, stopping, and monitoring Resin processes. This makes it an important reliability and management feature for Resin.
As a result of this change in architecture, we introduced several changes to the configuration and start up procedure. First, you can configure almost all aspects of your Resin servers in the Resin configuration (resin.xml). This includes not only network aspects like HTTP ports, IP addresses, etc., but JVM settings as well. In other words, you don’t have to write your own shell scripts with separate JVM configurations for each Resin server; all of that information is contained within the unified resin.xml file. For users running multiple Resin instances per machine, you can use a single watchdog to manage and monitor all of those instances.
What the Watchdog is not
The Watchdog is really very small and entirely dedicated to starting and stopping the Resin instances. The only monitoring it does of Resin instances is to see if they are started or stopped. If you’re familiar with other application servers that have more complex monitoring and start up processes, this might be a bit confusing (e.g. WebLogic’s Node Managers). When you log into the Resin administration console, you’re actually logging into an application on the Resin server, not the Watchdog. The Watchdog doesn’t open an HTTP port or serve any applications.
This architecture is by design and mainly for security reasons. In Unix/Linux we can run Resin instances as a non-root user by having it bind to protected ports (e.g. 80) as root, then dropping permissions. The Watchdog needs to run as root for the binding part, so keeping it smaller makes it more easily inspected for potential problems. However, if you’d like to see more features in the Watchdog, let us know by commenting below.
Starting Resin and managing the Watchdog
To start Resin manually from the command line, you just need to run the resin.jar in the “lib” directory of your Resin home. There was a change in the early 4.0’s that you need to pass a command to resin.jar to start, meaning the default behavior to run on the console has changed. Now you need to pass the “console” command:
$ java -jar $RESIN_HOME/lib/resin.jar console
I don’t think many users ran Resin this way anyhow, but we do it a lot internally for debugging and development, so you might like to know about it.
Note that the Watchdog is all about process management too, so you shouldn’t background a Resin instance run with the “console” command. If you do that, you’ll just lose your standard output and error. By running Resin with the “start” command, your logs will go to the right places and actually do rollover if you’ve specified that.
Here’s a quick summary of the commands you can use with Resin:
| Command | Description | Changed behavior |
|---|---|---|
| console | Starts Resin with output to the console | The previous behavior way to run on the console was to run the resin.jar without any command. The “console” command is now required. |
| start | Starts a new Resin instance in the background. If the Watchdog is already running, this command simply contacts the Watchdog to start that new Resin instance. If the Watchdog is not already running, it is started and then it starts the Resin instance | No change |
| restart | Restart a running Resin instance. Will start the Resin instance as well if it’s not already running. | No change |
| stop | Stops a running Resin instance | Does not stop the Watchdog in any situation. In 3.1, the “stop” command would also stop the Watchdog if there were no more running Resin instances after the target instance was stopped. In 4.0, the Watchdog will continue to run even without Resin instances. |
| kill | Forces a Resin instance to stop | Does not stop the Watchdog (similar to “stop”) |
| shutdown | Stops all running Resin instances and the Watchdog | New command |
| status | Shows the running status of all the running Resin instances | The process id is now shown for scripts that might want this information |
In case you missed it, you should note that you need to run the “shutdown” command to stop the Watchdog. This behavior is new in Resin 4.0.
Authorization problems
One problem you might have run into while setting up new Resin deployment is that you get a com.caucho.bam.NotAuthorizedException after setting up a new password. If you’re running Resin in the background, set up a new password, and add it to your configuration while the Watchdog is running, you’ll get this error. The reason is that the Watchdog loads the password configuration only once on its startup while Resin loads it every time. In other words, the two can get out of sync when you add passwords at runtime. The solution is to stop Resin when you’re adding, removing, or changing users or passwords in the AdminAuthenticator.
Future directions for the Watchdog
Some ideas we have for the Watchdog are to make it capable of launching Resin instances in response to a remote command (mainly for dynamic servers), integrate Watchdogs in a network, and add monitoring. We’ve already got a feature bug for starting the Watchdog without any Resin instances. This was mainly to fix an issue with the Mac OS X launcher, but it could have uses in automated management as well.
Do any of these changes sound particularly appealing? Are we missing something? Comment below or file a feature bug on our bug tracker.