Differences

This shows you the differences between two versions of the page.

--- daq:daq [2015/08/31 10:20]
weisshaa
+++ daq:daq [2023/11/28 10:47]
stephen
@@ Line 1: / Line 1: @@
-======= This is status page from 2015 campaign ======
+======= FRIB online build window ======
-Status page from former S800-GRETINA campaign in 2012 can be found as
+Issue the following command prior to starting the ReadoutGUI: export COINCIDENCE_TICKS=300
-https://wikihost.nscl.msu.edu/gretina/doku.php/daq:daq_2012
-====== Look Up table (LH target) 4/3/2013-- ======
+======= Turn on/off Mode3 Data collection ======
+  * To start/stop mode3 data collection on the EPICS GUI page click the button Cluster. In the newly opened window click the button **2 only** for mode2 data only or the **2 and 3** for both mode2 and mode3 data.
+  * Alternatively in a terminal on a2.gam run the command **"caput Data_CS_Save2And3 0/1"** to turn mode3 collection off/on.
+  * After doing this the decomp scripts should be modified as running the syncrofix command could overwrite this.
+  * For GRETINA standalone the script to modify is is **grdecompSA.sh** and for GRETINA-S800, **grdecompParticleGamma.sh**.
+  * For both scripts change the line **"caput Data_CS_Save2And3 0/1"** to turn mode3 collection off/on
+======= New in 2022 ======
+start run window is obsolete. Two scripts are available for changing the data directory. Daq needs to be stopped, directories need to exist:
+  * changeDataDir_maintainRunNum  <data dir>  : keeps the Run number the same, for when we need to move physics experiment data collection to another disk
+  * changeDataDir_resetRunNum <data dir>  : finds the lowest available Run number without any larger RunDir #s
-^  Spectcl  ^  CRY id  ^  Qp  ^
-^  00  |  26  |  Q2p3  |
-^  01  |  30  |  Q6p3  |
-^  02  |  25  |  Q2p2  |
-^  03  |  29  |  Q6p2  |
-^  04  |  27  |  Q2p4  |
-^  05  |  31  |  Q6p4  |
-^  06  |  24  |  Q2p1  |
-^  07  |  28  |  Q6p1  |
-^  08  |  63  |  Q1p4  |
-^  09  |  47  |  Q5p4  |
-^  10  |  59  |  Q3p4  |
-^  11  |  51  |  Q4p4  |
-^  12  |  60  |  Q1p1  |
-^  13  |  44  |  Q5p1  |
-^  14  |  58  |  Q3p3  |
-^  15  |  50  |  Q4p3  |
-^  16  |  62  |  Q1p2  |
-^  17  |  46  |  Q5p2  |
-^  18  |  56  |  Q3p1  |
-^  19  |  48  |  Q4p1  |
-^  20  |  61  |  Q1p2  |
-^  21  |  45  |  Q5p2  |
-^  22  |  57  |  Q3p2  |
-^  23  |  49  |  Q4p2  |
-^  24  |  88  |  Q7p1  |
-^  25  |  91  |  Q7p4  |
-^  26  |  89  |  Q7p2  |
-^  27  |  90  |  Q7p3  |
-<file>
-LHconfiguration.tcl:
-# predef's for GRETINA
-# in LH target configuration
-# most fw, crystal 3 in Q2 and Q6
-treevariable -set g.xtal.00.crystal_id 26
-treevariable -set g.xtal.01.crystal_id 30
-# crystal 2
-treevariable -set g.xtal.02.crystal_id 25
-treevariable -set g.xtal.03.crystal_id 29
-#crystal 4
-treevariable -set g.xtal.04.crystal_id 27
-treevariable -set g.xtal.05.crystal_id 31
-#crystal 1
-treevariable -set g.xtal.06.crystal_id 24
-treevariable -set g.xtal.07.crystal_id 28
-# 90 deg, Q1, 5, and Q3, 4 crystal 1
-treevariable -set g.xtal.08.crystal_id 63
-treevariable -set g.xtal.09.crystal_id 47
-treevariable -set g.xtal.10.crystal_id 59
-treevariable -set g.xtal.11.crystal_id 51
-# Q1, 5 xtal 1, Q3, 4 xtal 3
-treevariable -set g.xtal.12.crystal_id 60
-treevariable -set g.xtal.13.crystal_id 44
-treevariable -set g.xtal.14.crystal_id 58
-treevariable -set g.xtal.15.crystal_id 50
-# Q1, 5 xtal 3, Q3, 4 xtal 1
-treevariable -set g.xtal.16.crystal_id 62
-treevariable -set g.xtal.17.crystal_id 46
-treevariable -set g.xtal.18.crystal_id 56
-treevariable -set g.xtal.19.crystal_id 48
-# Q1, 5 xtal 2, Q3, 4 xtal 2
-treevariable -set g.xtal.20.crystal_id 61
-treevariable -set g.xtal.21.crystal_id 45
-treevariable -set g.xtal.22.crystal_id 57
-treevariable -set g.xtal.23.crystal_id 49
-# Q7 xtal 1, 4, 2, 3
-treevariable -set g.xtal.24.crystal_id 88
-treevariable -set g.xtal.25.crystal_id 91
-treevariable -set g.xtal.26.crystal_id 89
-treevariable -set g.xtal.27.crystal_id 90
-</file>
-====== Look Up table 7/19/2012--4/1/2013 ========
-^  Spectcl  ^  CRY id  ^  Qp  ^
-^  00  |	26	|  Q2p3  |
-^  01  |  30  |  Q6p3  |
-^  02  |  34  |  Q4p3  |
-^  03  |  38  |  Q5p3  |
-^  04  |  25  |  Q2p2  |
-^  05  |  29  |  Q6p2  |
-^  06  |  33  |  Q4p2  |
-^  07  |  37  |  Q5p2  |
-^  08  |  27  |  Q2p4  |
-^  09  |  31  |  Q6p4  |
-^  10  |  35  |  Q4p4  |
-^  11  |  39  |  Q5p4  |
-^  12  |  24  |  Q2p1  |
-^  13  |  28  |  Q6p1  |
-^  14  |  32  |  Q4p1  |
-^  15  |  36  |  Q5p1  |
-^  16  |  63  |  Q1p4  |
-^  17  |  67  |  Q3p4  |
-^  18  |  71  |  Q7p4  |
-^  19  |  60  |  Q1p1  |
-^  20  |  66  |  Q3p3  |
-^  21  |  68  |  Q7p1  |
-^  22  |  62  |  Q1p3  |
-^  23  |  64  |  Q3p1  |
-^  24  |  70  |  Q7p3  |
-^  25  |  61  |  Q1p2  |
-^  26  |  65  |  Q3p2  |
-^  27  |  69  |  Q7p2  |
-GRETINA DAQ system
-====== HowTo for experimenter in charge ========
-  * Online SpecTcl [[:onlinespectcl|onlinespectcl]]
-====== Changelog ========
-ANY change to the DAQ system out here: [[:daqchange|daqchange]]
-**NOTE**: No change pass July 13th 2012 can be done without explicit
-announcement and approval (by Dirk).
-====== issues ========
-ANY issue/breakdown put here: [[:daqtrouble|daqtrouble]]
-ReadOutShell reports "S800 changed to inactive" [[:ReadOutShellS800Inactive|ReadOutShellS800Inactive]]
-====== start procedures ======
-  * [[GRETINA:GRETINA]]
-  *[[s800daq:s800daq]]
-  *[[source measurement:source measurement]]
-  * S800 monitor [[https://groups.nscl.msu.edu/opdevtech/wiki/index.php/S800_DAQ_status_tool|S800mon]]
-====== Documentation ======
-  * NSCL DAQ general [[http://docs.nscl.msu.edu/daq]]
-  * NSCL DAQ GRETINA (ReadOutShell scheme) [[http://docs.nscl.msu.edu/daq/gretinauserdocs.pdf]]
-  * S800 DAQ [[https://groups.nscl.msu.edu/opdevtech/wiki/index.php/Main_Page#Electronics_and_DAQ]]
-====== Trigger connection ======
-  * [[daq:triggerconnection|Trigger connection (Card 29)]]
-====== Status ======
-[[:status|status]]
+======= This is DAQ page from previous campaigns ======
+The DAQ page from the former S800-GRETINA campaign in 2012 can be found at
+https://wikihost.nscl.msu.edu/gretina/doku.php/daq:daq_2012
+  * S800-Gretina control [[docsS800Grt|docsS800Grt]]
+  * {{ :daq:ulm-e16011_names.png?linkonly |S800 Trigger GUI}}
+======= Troubleshoot ======
+  * S800-Gretina not correlated: Check CC setfile ~s800/converged_daq/Scripts/CC0105Begin.tcl for extsynch and extclock
+  * S800 is losing events. Check the network speed of spdaq32 by /sbin/ethtool eth0 and /sbin/ethtool eth1 , both should show speed as 1000Mb/s, if one shows 100Mb/s then the network is too slow and S800 cannot send data to GEB fast enough which will cause event loss happen.
+  * Check the ringbuffer status, $DAQBIN/ringbuffer status - -host=spdaq32 - -all , if the free colum becomes zero, DAQ related people needs to be notified for diagnostics.
+======= Decomp Information ======
+  * [[easy_decomp_calibrations|Easy Gretina Decomp File Creation]]
+  * [[decomp_calibrations|Gretina Decomp Calibrations Explanation]]
-==== File Structure on GRETINA cluster / files to be kept ====
+======= Gretina Computing =======
+Gretina computer farm is maintained by LBNL computer group and local NSCL IT department. Nonetheless I put documents here, but we do NOT do anything to the GRETINA computers!
-   Here offline decomp data gets stored:
-    /global/data1a/gretina/user1
-    /global/data1a/gretina/user2
-    /global/data1a/gretina/user2_1
-    /global/data1a/gretina/user4
-   Base + data to create them:
+  * May 2019, documentation provided from Jackie: [[https://docs.google.com/document/d/1Aq5NkuEKBc4_5fx-95d2RaSZM08VQwVleYWr42QTsFg/edit?invite=CNr_nogM&ts=5cd488d9|Gretina System Documentation]]
-   /global/data1a/
+  * Instruction to shut down switches for NSCL power shutdown 12/5/2015 {{:daq:gretinaswitchesshutdown.pdf|}}
-   drwxrwxrwx  8 mario   bin   4096 Mar 27 12:52 q1a1
-   drwxr-xr-x  6 mario   bin   4096 Nov 24  2010 q1a3
-   drwxrwxrwx  9 mario   bin   4096 Mar 27 12:53 q1b1
-   drwxr-xr-x  7 mario   bin   4096 Mar 27 12:53 q1b2
-   drwxr-xr-x  9 mario   bin   4096 Mar 27 12:55 q2a5
-   drwxr-xr-x  9 mario   bin   4096 Mar 27 12:57 q2a6
-   drwxrwxrwx  7 mkp     bin   4096 Mar 27 13:01 q2b4
-   drwxr-xr-x  7 mkp     bin   4096 Mar 27 13:03 q2b6
-   drwxr-xr-x  5 mario   bin   4096 Mar 27 13:05 q3a14
-   drwxr-xr-x  6 mario   bin   4096 Mar 27 13:06 q3a7
-   drwxr-xr-x  5 mario   bin   4096 Mar 27 13:08 q4a4
-   drwxr-xr-x  5 mario   bin   4096 Mar 27 13:09 q4a8
-   drwxr-xr-x  5 mario   bin   4096 Mar 27 13:11 q4b7
-   drwxr-xr-x  6 mario   bin   4096 Mar 29 14:00 q4b8
-   /global/data1b/
-   drwxr-xr-x  5 mario   bin   4096 Mar 27 14:26 q3b14
-   drwxr-xr-x  5 mario   bin   4096 Mar 27 14:27 q3b5
-   drwxr-xr-x  5 gretina bin   4096 Mar  6 18:07 q5a10
-   drwxr-xr-x  5 gretina bin   4096 Mar  6 18:17 q5a12
-   drwxr-xr-x  5 gretina bin   4096 Mar  7 13:14 q5b10
-   drwxr-xr-x  5 gretina bin   4096 Mar  6 18:27 q5b3
-   drwxr-xr-x  5 gretina bin   4096 Mar 27 14:29 q6a15
-   drwxr-xr-x  5 gretina bin   4096 Mar 27 14:29 q6a3
-   drwxr-xr-x  5 gretina bin   4096 Jun 10  2011 q6b11
-   drwxr-xr-x  5 gretina bin   4096 Jun 10  2011 q6b9
-   drwxr-xr-x  3 gretina bin   4096 Mar 27 14:31 q7a11
-   drwxr-xr-x  5 gretina bin   4096 Mar 27 14:32 q7a9
-   drwxr-xr-x  5 gretina bin   4096 Mar 27 14:33 q7b13
-   drwxr-xr-x  5 gretina bin   4096 Mar 27 14:33 q7b15
-G	DNLapril2012:   DNL data of digitizers
-====== Configuration files for decompostion ======
+======= NSCL GRETINA LaBr Array =======
-The configuration for the decomposition is here: https://wikihost.nscl.msu.edu/gretina/doku.php/daq:decomp
-====== Positioning of the quads ======
+    * [[fastscint_daq| FastScint DAQ Quick Start]]
-When a position of a detector is changed the following has to be updated:
+    * [[fastscint_daq_long | FastScint DAQ Details]]
-  * CS_DataId in https://wikihost.nscl.msu.edu/gretina/doku.php/daq:decomp
-  * CS_GblId in /global/devel/boot/vmeArray/vme*.cmd --- The Id is given by  hole_number * 16 + position_in_quad * 4 + board_number
-  * (GTSort_shack/DetNum.dat --- self explanatory) <-- only if GTSort_shack is still in use for online spectra.
-  * Changes need to be made in non-linearity correction code from Heather (due to hardcoded detector positions). The file to be edited is in <DAQversion#>/gretClust/gretClustApp/src/DecompIF.C   In this file, partway through that are a number of defines for the hole numbers for Q1-Q7.  These need to be changed to reflect the correct hole positions.  Then the DAQ codes needs to be recompiled (as user gretina, on dogs.gam), in the directory with the file, and two levels up (in the gretClust directory).  Ask Mario/Heather for help if necessary. Problem will show up as RED Run Status on node display window. (Reason: ID in the code comes from hole#, if that is changed, wrong ID is calculated --> SegFault).
-====== How to replace a broken node with a spare one ======
+======= Misc 2019 ======
-To replace a node of the cluster (e.g. n000x) with a spare one (e.g. n000y) follow the following procedure:
+  * Dual homed NSCL machines are spdaq32 and spdaq07
-In the file /global/devel/gretTop/11-1/gretClust/bin/linux-x86/setup7modulesRaw.sh and in the file /global/devel/gretTop/11-1/gretClust/bin/linux-x86/setupEndtoEnd28.sh
-  * replace node000x_CS_Program? with node000y_CS_Program?
-  * from the "GRETINA CLUSTER node allocation" window identify the name of the Decomp process (middle click on the crystal number) and replace it in the files
-  * change the n000x to offline and the n000y to online using the variables Decomp???_CS_Enable
+====== miscellaneous ======
+  * Validation/Trigger window width is set by Dig116_CS_X55WidthP, Dig116_CS_X55WidthN. N/P stands for positive/negative maximum difference relative to Trigger timestamp which will be read out.
+  * Bringing up just the EPICs screen: in gretina home **./startGRETINA_EPICSdisplay**
+  * Set trigger multiplicity: **caput Trig0_CS_FSCThr n** (n=0,1,2,3... for mult 1,2,3,4...)
+  * See trigger pulse width: **CheckLED.sh**
+  * Change trigger pulse width: **SETLEDWIN <# of 20ns ticks>**
+  * Check timestamps, report boards that are out-of-sync: **CheckTS**
+  * Force timestamps to reset, DO NOT USE DURING RUN!!!: **SINGLEIMPSYNC**
+  * Warning script: run on a2 as gretina in ~/scripts: **./warningMessage4.sh **
+  * Check decomp stats  ** recordDecomp.sh | head -n50 **
+  * [[Flash digitizer firmware]]
+  * Check digitizer firmware version: on a2 or a1 terminal, type: **DIGSNIDFW** ,all digitizers should have same version firmware
+  * [[Replace digitizer with different color ones]]
+  * [[Replace fuses of digitizer]]
+  * [[No valid decomposed events]]
+  * Verify connecedt digitizers, at IOC interface, type **dbior**, it will print all identified digitizers within one bank
+  * VME related scripts are located at /global/devel/boot/vmeArray
+  * Check connected PVs, at IOC interface, type "casr"
+  * [[ Move detectors to new Hole position ]]
-====== DAQ cold restart ======
+====== Reboot a2 ======
-  * Switch off all the vme crates.
+  * If a2 is rebooted due to system failure, two scripts need to be brought up
-  * wait 30 s
+    * from ~gretina: nohup scripts/warningMessage4.sh &
-  * Switch on the vme crate of the trigger cards.
+    * from ~gretina/scripts/watcherdir: nohup watcher &
-  * wait till the red lights over the trigger board are on (~ 3 minutes)
+ The first is the alarm script that talks to users.  The second is responsible for two things: writing record{Counters,Decomp,Settings} in the current run directory and cleaning up cluster process issues (usually restarting broken processes) soon after starting or stopping runs.
-  * Switch on all the other vme crates
-  * wait till the green blinking lights appears on all the digitizers (~ 3 minutes)
-  * start the daq and check the synchronization (alarm window or TS window). If few of the digitizers are out of sync recover them using the "reboot IOCs button", otherwise power-cycle on their crate
+  * Switch to a1
+    * If a2 cannot be brought back. Gretina DAQ can still run on a1. In the file ~/.profile , environment variable **DAQSERVER** needs changed to be a1.gam . This parameter is used to restrict gretina daq critical scripts to run on specific machine.

Trace:

Differences

Search

Toolbox

QR Code