Add example of monitoring using the influxDB monitoring library, with task unpacker

requested review from @j.decuveland

assigned to @f.uhlig

changed the description

added 1 commit

83e0d71a - Add example of monitoring using the influxDB monitoring library, with task unpacker

Compare with previous version

added 1 commit

462bc671 - Add example of monitoring using the influxDB monitoring library, with task unpacker

Compare with previous version

As example: screenshot of the included unpacker dashboard

Dear @f.uhlig, @v.friese, @p.-a.loizeau, @v.singhal,

you have been identified as code owner of at least one file which was changed with this merge request.

Please check the changes and approve them or request changes.

added CodeOwners label

Two minor remarks, as above.

More generally, what is the relation of the CbmRecoUnpack and CbmReco classes? The cbmreco_fairrun binary, which already instantiates the Monitoring class, but does not use it yet, currently uses the latter -- could that one be parameterized to replace the dedicated CbmRecoUnpack class? Or is it intended that we port the functionality over?

Otherwise, I think this is indeed how the monitoring interface class is generally meant to be used.

approved this merge request

CbmRecoUnpack is older (July 2021) and was implemented in the context of the renewed unpacking scheme, activity mostly by Pascal Raisig at that time. This class is hooked into CbmSourceTsArchive and is probably still in use when running the unpacking from ROOT macro level. It still includes monitoring with histogramming and the like.

CbmRecoUnpack does not derive from FairTask, so tries to go towards a more framework-independent implementation. However, it still links to ROOT, mainly for the histogramming stuff.

CbmReco is the main instance for the entire online processing chain, comprising for the time being unpacking, triggering and event building. This is used from the binary cbmreco_fairrun. It wraps FairRun, i.e., the "offline" scheme. Unpacking in this context is represented by CbmTaskUnpack, deriving from FairTask. This class already uses the "modernized" unpacking algorithms from algo. So, in a sense it is a successor of CbmRecoUnpack.

Two consequences (in my view):

I would like to keep cbmreco_fairrun operational, also for the DC in July. At least until we have a replacement, evolving from Felix's developments. It is probably also instructive to compare performances, substantiating our suspicion that ROOT comes with too much penalty (in particular the I/O). That means the monitoring stuff in the MR would need to be ported to CbmTaskUnpack and/or CbmReco.
CbmTaskUnpack now instantiates and runs the unpackers, also implementing multi-threading (not on GPU, of course). CbmDevUnpack is basically the same, but not as FairTask, but as FairMQDevice. Both share still a large fraction of code. I think we should have a steering class like CbmUnpack (or just Unpack) in cbm::algo doing everything which is independent from the framework around (FairRun / FairMQ), so basically everything except the I/O. The task or device wrapper around that would be very lightweight, I think, and could easily be replaced by a hook into a yet different framework to run the entire thing.

Once a valid chain is established, we should definitely clean up and remove the then obsolete classes like CbmRecoUnpack.

added 21 commits

462bc671...2b882872 - 18 commits from branch computing:master
53aac11c - [unpack] In Much unpack, set log level to debug2 for nb masked chans in each MS
8a8bbae2 - Clang-format: add ordering rules for two more Flesnest headers
e2472c89 - Add example of monitoring using the influxDB monitoring library, with task unpacker

Compare with previous version

added 1 commit

e18591ad - Add example of monitoring using the influxDB monitoring library, with task unpacker

Compare with previous version

Summary comment: Most of the remaining discussion concerns general architectural considerations. The MR is working and does not impair existing functionality. I would therefore merge it.

added 9 commits

e18591ad...152e3a4c - 6 commits from branch computing:master
aac20ed1 - [unpack] In Much unpack, set log level to debug2 for nb masked chans in each MS
b84bf2c2 - Clang-format: add ordering rules for two more Flesnest headers
23a3199c - Add example of monitoring using the influxDB monitoring library, with task unpacker

Compare with previous version

resolved all threads

Add example of monitoring using the influxDB monitoring library, with task unpacker

Merge request reports

Activity