We’ve talked a lot about using MassTransit with RabbitMQ in a variety of scenarios on this blog. We talked about error handling, which is something you need to know how to implement in a real world MassTransit + RabbitMQ deployment. Another important aspect to a production deployment is monitoring. Lets take a look at how we can leverage the RabbitMQ Management HTTP API (installed with the RabbitMQ Management Plugin) to query our queues so we can monitor the health of our service bus.
Enabling the RabbitMQ Management Plugin
In A Simple MassTransit Publish/Subscribe Example, I detailed how to install RabbitMQ on Windows along with the RabbitMQ Management Plugin. You can refer to that post for the full details, but it’s important to know the RabbitMQ Management Plugin isn’t installed out of the box. One simple command will install it:
> rabbitmq-plugins enable rabbitmq_management
After a restart of the RabbitMQ service, visiting http://localhost:15672/ should bring you to the management GUI (default credentials are guest/guest). The HTTP API is available at http://localhost:15672/api/.
Simple REST API Call
Type the following URL into your browser to get a list of queues in your RabbitMQ instance:
http://localhost:15672/api/queues
You will need to supply the guest/guest credentials to access the API method. You should see the list of queues returned as JSON. If you want to get data for a single, specific queue, use the url format /api/queues/vhost/queue-name. The default vhost in RabbitMQ is “/” which URL encoded is %2F, so the the URL to retrieve info for the queue named MtPubSubExample_TestSubscriber would be http://localhost:15672/api/queues/%2F/MtPubSubExample_TestSubscriber and that would return the following JSON:
{
"memory":21856,
"messages":1,
"messages_details":{
"rate":0.0
},
"messages_ready":1,
"messages_ready_details":{
"rate":0.0
},
"messages_unacknowledged":0,
"messages_unacknowledged_details":{
"rate":0.0
},
"idle_since":"2014-08-16 9:30:50",
"consumer_utilisation":"",
"policy":"",
"exclusive_consumer_tag":"",
"consumers":0,
"backing_queue_status":{
"q1":0,
"q2":0,
"delta":[
"delta",
"undefined",
0,
"undefined"
],
"q3":1,
"q4":0,
"len":1,
"pending_acks":0,
"target_ram_count":"infinity",
"ram_msg_count":0,
"ram_ack_count":0,
"next_seq_id":180224,
"persistent_count":1,
"avg_ingress_rate":0.0,
"avg_egress_rate":0.0,
"avg_ack_ingress_rate":0.0,
"avg_ack_egress_rate":0.0
},
"state":"running",
"incoming":[
],
"deliveries":[
],
"consumer_details":[
],
"name":"MtPubSubExample_TestSubscriber",
"vhost":"/",
"durable":true,
"auto_delete":false,
"arguments":{
},
"node":"rabbit@PROWIN1"
}
In this snippet of JSON you can see the queue has a single message in it. Now, let’s see what we can do with this new knowledge to keep tabs on our queues.
What to Monitor
At a minimum, I would monitor the number of ready messages in your queues. What constitutes an alerting threshold obviously will depend on your application. If you expect to have fairly real-time processing of your messages, and have the necessary horsepower in your consumer(s) to keep up, then you might alert if the queue goes over 10 unprocessed messages. You just need to decide how sensitive you need it to be. I’ll show you how to get the data out.
Which queues should you monitor? Any queue you expect to receive messages likely warrants monitoring. This means, for example, in our simple publish/subscribe example, we would monitor the MtPubSubExample_TestSubscriber queue. We also will want to monitor the corresponding error queue (e.g. MtPubSubExample_TestSubscriber_error). In our customer portal example, we would monitor Loosely_CustomerPortal_Backend and Loosely_CustomerPortal_Backend_error.
How to Monitor
This is largely up to you and what monitoring tools you already have in place. Most monitoring tools such as Nagios, NewRelic, AlertSite, and System Center will have some notion of a custom counter or metric that you can have the system monitor and send alerts when they reach a certain threshold. If your selected monitoring tool has a RabbitMQ plugin, you’re in luck. Otherwise, you will need to write a small script to supply the data points to your monitoring system. Let’s take a look at how to do this in C#.
Checking Your RabbitMQ Queues in C#
The code for this sample is available on github at https://github.com/dprothero/Loosely.RabbitMq.Monitor. Clone that repository or follow along here to build the project from scratch. I am using Visual Studio 2013.
First, create a new C# Console project. We can call it Loosely.RabbitMq.Monitor and name the solution something like Loosely.RabbitMq.Utilities. Install the NuGet package Json.NET to your project. That will help us read the JSON response from the web service.
Add a new class file to your project and name it QueueInfo.cs. Delete the QueueInfo class declaration (leaving just the namespace declaration). Copy the JSON code above to your clipboard and then position your cursor within the namespace declaration in QueueInfo.cs. From the Edit menu, select Paste Special, and then Paste JSON as Classes. This will create C# class declarations that will follow your expected JSON. Rename the newly pasted class named Rootobject to QueueInfo. You should end up with the classes below in QueueInfo.cs:
namespace Loosely.RabbitMq.Monitor
{
public class QueueInfo
{
public int memory { get; set; }
public int messages { get; set; }
public Messages_Details messages_details { get; set; }
public int messages_ready { get; set; }
public Messages_Ready_Details messages_ready_details { get; set; }
public int messages_unacknowledged { get; set; }
public Messages_Unacknowledged_Details messages_unacknowledged_details { get; set; }
public string idle_since { get; set; }
public string consumer_utilisation { get; set; }
public string policy { get; set; }
public string exclusive_consumer_tag { get; set; }
public int consumers { get; set; }
public Backing_Queue_Status backing_queue_status { get; set; }
public string state { get; set; }
public object[] incoming { get; set; }
public object[] deliveries { get; set; }
public object[] consumer_details { get; set; }
public string name { get; set; }
public string vhost { get; set; }
public bool durable { get; set; }
public bool auto_delete { get; set; }
public Arguments arguments { get; set; }
public string node { get; set; }
}
public class Messages_Details
{
public float rate { get; set; }
}
public class Messages_Ready_Details
{
public float rate { get; set; }
}
public class Messages_Unacknowledged_Details
{
public float rate { get; set; }
}
public class Backing_Queue_Status
{
public int q1 { get; set; }
public int q2 { get; set; }
public object[] delta { get; set; }
public int q3 { get; set; }
public int q4 { get; set; }
public int len { get; set; }
public int pending_acks { get; set; }
public string target_ram_count { get; set; }
public int ram_msg_count { get; set; }
public int ram_ack_count { get; set; }
public int next_seq_id { get; set; }
public int persistent_count { get; set; }
public float avg_ingress_rate { get; set; }
public float avg_egress_rate { get; set; }
public float avg_ack_ingress_rate { get; set; }
public float avg_ack_egress_rate { get; set; }
}
public class Arguments
{
}
}
If you wanted, you could clean up the property and class names to better match C# naming conventions. Just be sure to add the necessary JsonProperty attribute so JSON.NET can deserialize the JSON into the correct object properties. For example:
[JsonProperty("avg_ack_egress_rate")]
public float AvgAckEgressRate { get; set; }
Note that JSON.NET is case insensitive when deserializing, so you don’t need these attributes if you are only adjusting the case of the identifier (e.g. “name” to “Name”).
Next, we could just hard-code the following in Program.cs to get the message count for the MtPubSubExample_TestSubscriber queue:
using Newtonsoft.Json;
using System;
using System.Net;
namespace Loosely.RabbitMq.Monitor
{
class Program
{
static void Main(string[] args)
{
var client = new WebClient();
client.Credentials = new NetworkCredential("guest", "guest");
var data = client.DownloadString("http://localhost:15672/api/queues/%2F/MtPubSubExample_TestSubscriber");
var queueInfo = JsonConvert.DeserializeObject<QueueInfo>(data);
Console.WriteLine("Queue: " + queueInfo.Name);
Console.WriteLine("Queue Depth: " + queueInfo.MessagesReady.ToString());
Console.Write("\r\nPress any key to continue.");
Console.ReadKey();
}
}
}
The code should be pretty straight forward. We make a request to the HTTP API, deserialize the JSON data into a plain old C# object (POCO), and then output the properties we are interested in. However, to make this really useful, we should refactor this code so it can be command line driven. Then, we can simply pass parameters on the command line to tell the program the instance of RabbitMQ to monitor (base URL for the API), the credentials to connect to the server, and the queue we want to interrogate.
For command line argument parsing to a console application, I like to use the NDesk.Options NuGet package. It makes supporting command line parameters (-x or –xxxx or /XXXX) super easy. Add the NDesk.Options package to your project. With that added, we can modify our Program.cs file to the following:
using NDesk.Options;
using Newtonsoft.Json;
using System;
using System.Net;
namespace Loosely.RabbitMq.Monitor
{
class Program
{
static void Main(string[] args)
{
string baseUrl = "http://localhost:15672/api",
userName = "guest",
password = "guest",
queueName = null,
vhost = "/";
var p = new OptionSet () {
{ "u|baseUrl=", v => baseUrl = v },
{ "U|user=", v => userName = v },
{ "p|password=", v => password = v },
{ "q|queue=", v => queueName = v },
{ "v|vhost=", v => vhost = v }
};
p.Parse(args);
var client = new WebClient();
if(userName != null)
client.Credentials = new NetworkCredential(userName, password);
var url = baseUrl + "/queues/" + Uri.EscapeDataString(vhost) + "/" + Uri.EscapeDataString(queueName);
var data = client.DownloadString(url);
var queueInfo = JsonConvert.DeserializeObject<QueueInfo>(data);
// Here is where we report the message count to our monitoring system.
// Replace the console output/wait with your code.
Console.WriteLine("Queue: " + queueInfo.Name);
Console.WriteLine("Queue Depth: " + queueInfo.MessagesReady.ToString());
Console.Write("\r\nPress any key to continue.");
Console.ReadKey();
}
}
}
To be able to run from within Visual Studio, you can try out different command line arguments by going to the Debug project properties page:
Conclusion
As you can see, it is fairly easy to interrogate RabbitMQ to get information about the status of its queues (as well as any number of other objects). If you want to make extensive use of the HTTP API, I would suggest taking a look at Mike Hadlow’s EasyNetQ.Client.Management library, available on NuGet. This is a complete C# wrapper around the HTTP API, making it trivial to work with the API from C#.
I hope you found value in this blog post. Please don’t hesitate to contact me if you have any questions or suggestions for future blog posts. Until then…
Can the queue depth be obtained using MassTransit instead of the RabbitMQ api http call?
I do not believe so, no.