Document toolboxDocument toolbox

Vidispine

How to expose number of messages in the IndexQueue

What is IndexQueue and why would you want to view the messages in there?

Some operation leads to modification of metadata, the operation updates the database in the same transaction.

That means that when the API call returns, the database is already updated.
But the operation does not update Solr directly. instead, the operation sends a message to the IndexQueue that item X (or items, or collections or other) has new metadata.


The indexcruncher reads from the IndexQueue (but *not* in the transaction), so this can be a slower process. 

For every message (batch of messages) read from the queue, Solr is updated.
So, if Solr is slow or indexcruncher is dead for some reason, the number of messages in IndexQueue will grow.

 

There are multiple ways to do this, depending on how your system is running.

Here are the different ways described.

If you are using standalone ActiveMQ, there is a Web Console you can use;

http://activemq.apache.org/web-console.html

 

If you are using embedded ActiveMQ, you will need to use  jmx=true , and then view the properties using VisualVM.

Read more about this in the link below;

http://activemq.apache.org/broker-uri.html

 

We have written a small Groovy script that can be used to retrieve the queue size from embedded ActiveMQ;

import java.lang.management.* import javax.management.remote.JMXConnectorFactory as JmxFactory import javax.management.remote.JMXServiceURL as JmxUrl def serverUrl = 'service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi' def beanName = 'org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Queue,destinationName=IndexQueue' def server = JmxFactory.connect(new JmxUrl(serverUrl)).MBeanServerConnection def queue = new GroovyMBean(server, beanName) println "Current size: "+queue.QueueSize