Draft: MQ Device for Eventbuilder Monitoring

Added a second output channel to CbmDevBuildEvents through which a second instance of the message which carries the constructed events is sent. This channel is currently configured to run in "pub" mode.

Added a new class CbmDevBuildEventsMonitor, which receives this new message. It then constructs histograms containing "tTrigger - tDigi", once for all subsystems and then for each subsystem individually. These histograms are sent to the histogramm server.

Open issues:

• Right now, the serialization is run twice, as it apparently is not possible to copy FairMQParts objects. This is inefficient and can perhaps be improved.
• The histogramm server can receive configuration data for histograms and canvases it receives. These are sent through a std::string which is set in InitHistoContainers() for each histogram and canvas. Currently these are empty. Probably a folder name should be added here.

@p.-a.loizeau Can you comment on the above issues?

Also: So far, it has only been tested that the histogram server receives messages, but the output has not been inspected in a web browser.

@v.friese @f.uhlig

On edit: Both issues above have been addressed. The output is copied message-wise, which is very fast. Removing the WIP flag, as I expect things to work as they should now.

@p.-a.loizeau Could you remind me how I can log into the histogram server from a web browser? I still would like to test this.

changed the description

Dear @f.uhlig, @v.friese, @p.-a.loizeau,

you have been identified as code owner of at least one file which was changed with this merge request.

Please check the changes and approve them or request changes.

added CodeOwners label

added 8 commits

• 6a49d012 - 1 commit from branch computing:master
• 6b999e15 - Added second output channel in ``pub'' mode to CbmDevBuildEvents.
• 139efb71 - Added class CbmDevBuildEventsMonitor.
• 4a325382 - CbmDevBuildEventsMonitor: Now receives messages from CbmDevBuildEvents. Added histogram and canvas.
• aa117f34 - CbmDevBuildEventsMonitor: Can communicate with histogram server.
• 74774352 - CbmDevBuildEventsMonitor: Fills tEvent - tDigi histogram for Sts.
• 31e4a79c - CbmDevBuildEventsMonitor: Added histograms for all subsystems.
• 516ae437 - CbmDevBuildEvents: Double serialization of events is now avoided.

Compare with previous version

changed title from WIP: MQ Device for Eventbuilder Monitoring to MQ Device for Eventbuilder Monitoring

changed the description

@d.smith if you are running on your own computer, you should find it just by going to localhost:8081 in your browser (port in your startup script). If it is remote, you will need an ssh tunnel similar to ssh -N': ssh -N -L 8080:en13:8080 mfles-ext (I have shorts for my user login and hops in my ssh config file)

It may be empty at the beginning, as the emission of the plots is done

• only after processing a TS
• and by default every `5 s or 100 TS, whichever happens first)

The thing is that the histos are registered only when they are received for the first time, so until at least one TS passes the bar of the 5s since startup, there is not list.

In this case you just need to refresh the page in your browser a few times until the list fills up with all your expected plots and canvases. Afterward they are merged so the same histo just gets updated in the browser if you tick the "refresh" box.

added 9 commits

• a93dafb5 - 1 commit from branch computing:master
• 0505ab93 - Added second output channel in ``pub'' mode to CbmDevBuildEvents.
• ee249b29 - Added class CbmDevBuildEventsMonitor.
• 0e70a7f9 - CbmDevBuildEventsMonitor: Now receives messages from CbmDevBuildEvents. Added histogram and canvas.
• 0385abb3 - CbmDevBuildEventsMonitor: Can communicate with histogram server.
• 8eff2c23 - CbmDevBuildEventsMonitor: Fills tEvent - tDigi histogram for Sts.
• 115ef109 - CbmDevBuildEventsMonitor: Added histograms for all subsystems.
• 8034cd23 - CbmDevBuildEvents: Double serialization of events is now avoided.
• 9660c2ac - CbmDevBuildEventsMonitor: Fixed canvas initialization. Moved histograms to a subfolder.

Compare with previous version

@p.-a.loizeau Thanks. I managed to view the output of the device in a browser. It turned out that the canvas was not displayed correctly, basically the issue that you mentioned in the meeting. I had to move the Draw() call to the Init() function, then it worked. I also moved the histograms to a proper subfolder.

I think this is basically done now. Only STS data is displayed currently, but that is because the Unpacker device doesn't support other systems yet. Fixing this is a separate MR in my opinion.

mentioned in merge request !902 (merged)

assigned to @v.friese

requested review from @v.friese

As discussed on Wednesday in the meeting the MR should wait till Volker is back for his review.

added 234 commits

• 9660c2ac...31d5721c - 226 commits from branch computing:master
• 8f90ecc4 - Added second output channel in ``pub'' mode to CbmDevBuildEvents.
• 4b811929 - Added class CbmDevBuildEventsMonitor.
• 5d5a1405 - CbmDevBuildEventsMonitor: Now receives messages from CbmDevBuildEvents. Added histogram and canvas.
• 238007ae - CbmDevBuildEventsMonitor: Can communicate with histogram server.
• 5f7a8db3 - CbmDevBuildEventsMonitor: Fills tEvent - tDigi histogram for Sts.
• c1e2ef14 - CbmDevBuildEventsMonitor: Added histograms for all subsystems.
• b8ba2d59 - CbmDevBuildEvents: Double serialization of events is now avoided.
• 867944d2 - CbmDevBuildEventsMonitor: Fixed canvas initialization. Moved histograms to a subfolder.

Compare with previous version

@f.uhlig I just updated the changes to CMakeLists.txt, to match the new CMake style and solve the rebase conflict. Can you take a look and confirm that this is the proper new way of doing things?

added 130 commits

• 867944d2...b2439e1a - 122 commits from branch computing:master
• 6a42df6d - Added second output channel in ``pub'' mode to CbmDevBuildEvents.
• 8ab8683f - Added class CbmDevBuildEventsMonitor.
• d1a2479a - CbmDevBuildEventsMonitor: Now receives messages from CbmDevBuildEvents. Added histogram and canvas.
• d98560e2 - CbmDevBuildEventsMonitor: Can communicate with histogram server.
• 2f92c14b - CbmDevBuildEventsMonitor: Fills tEvent - tDigi histogram for Sts.
• 009b9de5 - CbmDevBuildEventsMonitor: Added histograms for all subsystems.
• 36f166d7 - CbmDevBuildEvents: Double serialization of events is now avoided.
• b20226e0 - CbmDevBuildEventsMonitor: Fixed canvas initialization. Moved histograms to a subfolder.

Compare with previous version

mentioned in merge request !1093 (merged)

@d.smith This is pending since long because I am not sure whether it is still relevant. From the recent developments in algo/online, I does not seem that we will be using FairMQ for the monitoring - the main process communicates directly with the monitor instance (InfluxDB).

However, this here is not so much monitoring that QA, in my language. Also here, we go for a different concept, as dicsussed in the online meeting last Thursday.

On the other hand, the MR does not interfere or harm this approach, it may just add unnecessary code. What is your opinion?

@v.friese I think the QA functionality, i.e. the histogram generation, supplied in the device could be useful in a different context. I would hence propose to keep this MR open as a draft until we have a QA system in place, then move the histogram-related stuff over to the new system, then close this MR.

Fine with me!

marked this merge request as draft

added 1010 commits

• b20226e0...b5b9540c - 1002 commits from branch computing:master
• a30bc587 - Added second output channel in ``pub'' mode to CbmDevBuildEvents.
• 59256895 - Added class CbmDevBuildEventsMonitor.
• 64b807cc - CbmDevBuildEventsMonitor: Now receives messages from CbmDevBuildEvents. Added histogram and canvas.
• 667a1c5d - CbmDevBuildEventsMonitor: Can communicate with histogram server.
• 3122c3ff - CbmDevBuildEventsMonitor: Fills tEvent - tDigi histogram for Sts.
• a41c08d3 - CbmDevBuildEventsMonitor: Added histograms for all subsystems.
• 0443325c - CbmDevBuildEvents: Double serialization of events is now avoided.
• 9e10f8ff - CbmDevBuildEventsMonitor: Fixed canvas initialization. Moved histograms to a subfolder.

Compare with previous version

@v.friese I think we could close this MR: the histograms created here (t_seed - t_digi [ns]) are similar to the ones available for the event building stage in the histogram server associated with the online binary.

So I would guess Dominik did the porting which from his comment one year ago was the reason to keep this open as reference.