Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
daq:daq [2014/12/01 08:45]
weisshaa [Files/trees on GRETINA cluster to go] 		daq:daq [2024/02/12 05:23] (current)
weisshaa [FRIB online build window]
Line 1: Line 1:
-====== ​Look Up table (LH target) 4/​3/​2013-- ​======+======= ​FRIB online build window ​====== 
 +Issue the following command prior to starting the ReadoutGUI: export COINCIDENCE_TICKS=300 
 +(e.g. in bin/​startrdo)
  
-^  Spectcl ​ ^  CRY id  ^  Qp  ^ +======= Turn on/off Mode3 Data collection ====== 
-^  00  |  26  |  Q2p3  | +  ​* To start/stop mode3 data collection on the EPICS GUI page click the button Cluster. In the newly opened window click the button **2 only** for mode2 data only or the **2 and 3** for both mode2 and mode3 data. 
-^  01  |  30  |  Q6p3  | +  ​* Alternatively in a terminal on a2.gam run the command **"​caput Data_CS_Save2And3 0/1"** to turn mode3 collection off/on. 
-^  02  |  25  |  Q2p2  | +  ​* After doing this the decomp scripts should be modified as running the syncrofix command could overwrite this.  
-^  03  |  29  |  Q6p2  | +  ​* For GRETINA standalone the script to modify is is **grdecompSA.sh** and for GRETINA-S800,​ **grdecompParticleGamma.sh**. 
-^  04  |  27  |  Q2p4  | +  ​* For both scripts change the line **"​caput Data_CS_Save2And3 0/1"** to turn mode3 collection off/on 
-^  05  |  31  |  Q6p4  | +  
-^  06  |  24  |  Q2p1  | +======= New in 2022 ====== 
-^  07  |  28  |  Q6p1  | +start run window is obsolete. Two scripts are available for changing the data directory. Daq needs to be stopped, directories need to exist: ​ 
-^  08  |  63  |  Q1p4  | +  ​* changeDataDir_maintainRunNum ​ <data dir> ​ : keeps the Run number the same, for when we need to move physics experiment data collection to another disk 
-^  09  |  47  |  Q5p4  | +  ​* changeDataDir_resetRunNum <data dir> ​ : finds the lowest available Run number without any larger RunDir #s
-^  10  |  59  |  Q3p4  | +
-^  11  |  51  |  Q4p4  | +
-^  12  |  60  |  Q1p1  | +
-^  13  |  44  |  Q5p1  | +
-^  14  |  58  |  Q3p3  | +
-^  15  |  50  |  Q4p3  | +
-^  16  |  62  |  Q1p2  | +
-^  17  |  46  |  Q5p2  | +
-^  18  |  56  |  Q3p1  | +
-^  19  |  48  |  Q4p1  | +
-^  20  |  61  |  Q1p2  | +
-^  21  |  45  |  Q5p2  | +
-^  22  |  57  |  Q3p2  | +
-^  23  |  49  |  Q4p2  | +
-^  24  |  88  |  Q7p1  | +
-^  25  |  91  |  Q7p4  | +
-^  26  |  89  |  Q7p2  | +
-^  27  |  90  |  Q7p3  |+
  
-<​file>​ 
-LHconfiguration.tcl:​ 
  
-# predef'​s for GRETINA 
-# in LH target configuration 
-# most fw, crystal 3 in Q2 and Q6 
-treevariable -set g.xtal.00.crystal_id 26 
-treevariable -set g.xtal.01.crystal_id 30 
  
-# crystal 2 +======= This is DAQ page from previous campaigns ====== 
-treevariable ​-set g.xtal.02.crystal_id 25 +The DAQ page from the former S800-GRETINA campaign in 2012 can be found at 
-treevariable -set g.xtal.03.crystal_id 29+https://​wikihost.nscl.msu.edu/​gretina/​doku.php/​daq:​daq_2012
  
-#crystal 4 +  * S800-Gretina control [[docsS800Grt|docsS800Grt]] ​ 
-treevariable ​-set g.xtal.04.crystal_id 27 +  * {{ :daq:ulm-e16011_names.png?​linkonly |S800 Trigger GUI}} 
-treevariable ​-set g.xtal.05.crystal_id 31+======= Troubleshoot ====== 
 +  * S800-Gretina not correlated: Check CC setfile ~s800/​converged_daq/​Scripts/​CC0105Begin.tcl for extsynch and extclock 
 +  * S800 is losing events. Check the network speed of spdaq32 by /​sbin/​ethtool eth0 and /​sbin/​ethtool eth1 , both should show speed as 1000Mb/s, if one shows 100Mb/s then the network is too slow and S800 cannot send data to GEB fast enough which will cause event loss happen. 
 +  * Check the ringbuffer status, $DAQBIN/​ringbuffer status - -host=spdaq32 - -all , if the free colum becomes zero, DAQ related people needs to be notified for diagnostics.
  
-#crystal 1 +======= Decomp Information ====== 
-treevariable -set g.xtal.06.crystal_id 24 +  * [[easy_decomp_calibrations|Easy Gretina Decomp File Creation]] 
-treevariable -set g.xtal.07.crystal_id 28+  * [[decomp_calibrations|Gretina Decomp Calibrations Explanation]]
  
-# 90 deg, Q1, 5, and Q3, 4 crystal 1 +======= Gretina Computing ======= 
-treevariable -set g.xtal.08.crystal_id 63 +Gretina computer farm is maintained by LBNL computer group and local NSCL IT department. Nonetheless I put documents here, but we do NOT do anything to the GRETINA computers!
-treevariable -set g.xtal.09.crystal_id 47 +
-treevariable -set g.xtal.10.crystal_id 59 +
-treevariable -set g.xtal.11.crystal_id 51+
  
-# Q1, 5 xtal 1, Q3, 4 xtal 3 
-treevariable -set g.xtal.12.crystal_id 60 
-treevariable -set g.xtal.13.crystal_id 44 
-treevariable -set g.xtal.14.crystal_id 58 
-treevariable -set g.xtal.15.crystal_id 50 
  
-# Q1, 5 xtal 3, Q3, 4 xtal 1 +  * May 2019, documentation provided from Jackie: [[https://​docs.google.com/​document/​d/​1Aq5NkuEKBc4_5fx-95d2RaSZM08VQwVleYWr42QTsFg/​edit?​invite=CNr_nogM&​ts=5cd488d9|Gretina System Documentation]] 
-treevariable -set g.xtal.16.crystal_id 62 +  * Instruction to shut down switches for NSCL power shutdown 12/5/2015 {{:​daq:​gretinaswitchesshutdown.pdf|}}
-treevariable ​-set g.xtal.17.crystal_id 46 +
-treevariable -set g.xtal.18.crystal_id 56 +
-treevariable -set g.xtal.19.crystal_id 48+
  
-# Q1, 5 xtal 2, Q3, 4 xtal 2 +======= NSCL GRETINA LaBr Array =======
-treevariable -set g.xtal.20.crystal_id 61 +
-treevariable -set g.xtal.21.crystal_id 45 +
-treevariable -set g.xtal.22.crystal_id 57 +
-treevariable -set g.xtal.23.crystal_id 49+
  
 +    * [[fastscint_daq| FastScint DAQ Quick Start]]
 +    * [[fastscint_daq_long | FastScint DAQ Details]]
  
-# Q7 xtal 1, 4, 2, 3 +======= Misc 2019 ====== 
-treevariable -set g.xtal.24.crystal_id 88 +  * Dual homed NSCL machines are spdaq32 and spdaq07
-treevariable -set g.xtal.25.crystal_id 91 +
-treevariable -set g.xtal.26.crystal_id 89 +
-treevariable -set g.xtal.27.crystal_id 90+
  
-</​file>​ +====== ​miscellaneous ​====== 
-====== ​Look Up table 7/​19/​2012--4/​1/​2013 ========+  * Validation/​Trigger window width is set by Dig116_CS_X55WidthP,​ Dig116_CS_X55WidthN. N/P stands for positive/​negative maximum difference relative to Trigger timestamp which will be read out. 
 +  * Bringing up just the EPICs screen: in gretina home **./​startGRETINA_EPICSdisplay** 
 +  * Set trigger multiplicity:​ **caput Trig0_CS_FSCThr n** (n=0,1,2,3... for mult 1,​2,​3,​4...) 
 +  * See trigger pulse width: **CheckLED.sh** ​  
 +  * Change trigger pulse width: **SETLEDWIN <# of 20ns ticks>​** 
 +  * Check timestamps, report boards that are out-of-sync:​ **CheckTS** 
 +  * Force timestamps to reset, DO NOT USE DURING RUN!!!: **SINGLEIMPSYNC** 
 +  * Warning script: run on a2 as gretina in ~/scripts: **./​warningMessage4.sh ** 
 +  * Check decomp stats  ** recordDecomp.sh | head -n50 ** 
 +  * [[Flash digitizer firmware]] 
 +  * Check digitizer firmware version: on a2 or a1 terminal, type: **DIGSNIDFW** ,all digitizers should have same version firmware 
 +  * [[Replace digitizer with different color ones]] 
 +  * [[Replace fuses of digitizer]] 
 +  * [[No valid decomposed events]] 
 +  * Verify connecedt digitizers, at IOC interface, type **dbior**, it will print all identified digitizers within one bank 
 +  * VME related scripts are located at /​global/​devel/​boot/​vmeArray 
 +  * Check connected PVs, at IOC interface, type "​casr"​ 
 +  * [[ Move detectors to new Hole position ]]
  
-^  Spectcl ​ ^  CRY id  ^  Qp  ^  +====== ​Reboot a2 ====== 
-^  00  | 26 | ​ Q2p3  | +  * If a2 is rebooted due to system ​failure, two scripts need to be brought up 
-^  01  |  30  |  Q6p3  | +    * from ~gretina: nohup scripts/warningMessage4.sh & 
-^  02  |  34  |  Q4p3  | +    * from ~gretina/scripts/watcherdir: nohup watcher & 
-^  03  |  38  |  Q5p3  | + ​The ​first is the alarm script that talks to users.  The second ​is responsible ​for two things: writing record{Counters,Decomp,Settings} ​in the current run directory and cleaning ​up cluster process ​issues ​(usually restarting broken processes) soon after starting ​or stopping runs.
-^  04  |  25  |  Q2p2  | +
-^  05  |  29  |  Q6p2  | +
-^  06  |  33  |  Q4p2  | +
-^  07  |  37  |  Q5p2  | +
-^  08  |  27  |  Q2p4  | +
-^  09  |  31  |  Q6p4  | +
-^  10  |  35  |  Q4p4  | +
-^  11  |  39  |  Q5p4  | +
-^  12  |  24  |  Q2p1  | +
-^  13  |  28  |  Q6p1  | +
-^  14  |  32  |  Q4p1  | +
-^  15  |  36  |  Q5p1  | +
-^  16  |  63  |  Q1p4  | +
-^  17  |  67  |  Q3p4  | +
-^  18  |  71  |  Q7p4  | +
-^  19  |  60  |  Q1p1  | +
-^  20  |  66  |  Q3p3  | +
-^  21  |  68  |  Q7p1  | +
-^  22  |  62  |  Q1p3  | +
-^  23  |  64  |  Q3p1  | +
-^  24  |  70  |  Q7p3  | +
-^  25  |  61  |  Q1p2  | +
-^  26  |  65  |  Q3p2  | +
-^  27  |  69  |  Q7p2  | +
- +
- +
-GRETINA DAQ system +
- +
-====== ​HowTo for experimenter in charge ======== +
- +
-  * Online SpecTcl [[:​onlinespectcl|onlinespectcl]] +
- +
-====== Changelog ======== +
-ANY change ​to the DAQ system ​out here: [[:​daqchange|daqchange]] +
- +
-**NOTE**: No change pass July 13th 2012 can be done without explicit +
-announcement and approval (by Dirk).  +
- +
- +
-====== issues ======== +
-ANY issue/​breakdown put here: [[:​daqtrouble|daqtrouble]] +
- +
- +
-ReadOutShell reports "S800 changed to inactive"​ [[:​ReadOutShellS800Inactive|ReadOutShellS800Inactive]] +
- +
-====== start procedures ====== +
- +
-  ​* [[GRETINA:​GRETINA]] +
-  *[[s800daq:​s800daq]] +
-  *[[source measurement:​source measurement]] +
-  * S800 monitor [[https://groups.nscl.msu.edu/​opdevtech/​wiki/​index.php/​S800_DAQ_status_tool|S800mon]] +
- +
-====== Documentation ====== +
-  ​* NSCL DAQ general [[http://​docs.nscl.msu.edu/​daq]] +
-  * NSCL DAQ GRETINA (ReadOutShell scheme) [[http://​docs.nscl.msu.edu/​daq/​gretinauserdocs.pdf]] +
-  * S800 DAQ [[https://​groups.nscl.msu.edu/​opdevtech/​wiki/​index.php/​Main_Page#​Electronics_and_DAQ]] +
- +
-====== Trigger connection ====== +
- +
-  * [[daq:​triggerconnection|Trigger connection (Card 29)]] +
- +
- +
-====== Status ====== +
-[[:​status|status]] +
- +
- +
- +
- +
-==== File Structure on GRETINA cluster / files to be kept ==== +
- +
-   Here offline decomp data gets stored: +
-    /​global/​data1a/​gretina/user1 +
-    ​/global/​data1a/​gretina/​user2 +
-    /​global/​data1a/​gretina/​user2_1 +
-    /​global/​data1a/​gretina/​user4 +
- +
-   Base + data to create them: +
-   /​global/​data1a/​ +
-   ​drwxrwxrwx ​ 8 mario   ​bin ​  4096 Mar 27 12:52 q1a1 +
-   ​drwxr-xr-x ​ 6 mario   ​bin ​  4096 Nov 24  2010 q1a3 +
-   ​drwxrwxrwx ​ 9 mario   ​bin ​  4096 Mar 27 12:53 q1b1 +
-   ​drwxr-xr-x ​ 7 mario   ​bin ​  4096 Mar 27 12:53 q1b2 +
-   ​drwxr-xr-x ​ 9 mario   ​bin ​  4096 Mar 27 12:55 q2a5 +
-   ​drwxr-xr-x ​ 9 mario   ​bin ​  4096 Mar 27 12:57 q2a6 +
-   ​drwxrwxrwx ​ 7 mkp     ​bin ​  4096 Mar 27 13:01 q2b4 +
-   ​drwxr-xr-x ​ 7 mkp     ​bin ​  4096 Mar 27 13:03 q2b6 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:05 q3a14 +
-   ​drwxr-xr-x ​ 6 mario   ​bin ​  4096 Mar 27 13:06 q3a7 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:08 q4a4 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:09 q4a8 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:11 q4b7 +
-   ​drwxr-xr-x ​ 6 mario   ​bin ​  4096 Mar 29 14:00 q4b8 +
-   /​global/​data1b/​ +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 14:26 q3b14 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 14:27 q3b5 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  6 18:07 q5a10 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  6 18:17 q5a12 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  7 13:14 q5b10 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  6 18:27 q5b3 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:29 q6a15 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:29 q6a3 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Jun 10  2011 q6b11 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Jun 10  2011 q6b9 +
-   ​drwxr-xr-x ​ 3 gretina bin   4096 Mar 27 14:31 q7a11 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:32 q7a9 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:33 q7b13 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:33 q7b15 +
-   ​384G DNLapril2012: ​  DNL data of digitizers +
- +
-====== Configuration files for decompostion ====== +
-The configuration for the decomposition ​is here: https://​wikihost.nscl.msu.edu/​gretina/​doku.php/​daq:​decomp +
- +
-====== Positioning of the quads ====== +
-When a position of a detector is changed the following has to be updated: +
-  * CS_DataId in https://​wikihost.nscl.msu.edu/​gretina/​doku.php/​daq:​decomp +
-  * CS_GblId in /​global/​devel/​boot/​vmeArray/​vme*.cmd --- The Id is given by  hole_number * 16 + position_in_quad * 4 + board_number +
-  * (GTSort_shack/​DetNum.dat --- self explanatory) <-- only if GTSort_shack is still in use for online spectra. +
-  * Changes need to be made in non-linearity correction code from Heather (due to hardcoded detector positions). The file to be edited is in <​DAQversion#>/​gretClust/​gretClustApp/​src/​DecompIF.C ​  In this file, partway through that are a number of defines for the hole numbers for Q1-Q7. ​ These need to be changed to reflect the correct hole positions. ​ Then the DAQ codes needs to be recompiled (as user gretina, on dogs.gam), in the directory ​with the file, and two levels ​up (in the gretClust directory). ​ Ask Mario/​Heather for help if necessary. Problem will show up as RED Run Status on node display window. (Reason: ID in the code comes from hole#, if that is changed, wrong ID is calculated --> SegFault). +
- +
-====== How to replace a broken node with a spare one ====== +
-To replace a node of the cluster ​(e.g. n000x) with a spare one (e.g. n000y) follow the following procedure:​ +
-In the file /​global/​devel/​gretTop/​11-1/​gretClust/​bin/​linux-x86/​setup7modulesRaw.sh and in the file /​global/​devel/​gretTop/​11-1/​gretClust/​bin/​linux-x86/​setupEndtoEnd28.sh +
-  * replace node000x_CS_Program?​ with node000y_CS_Program?​ +
-  * from the "​GRETINA CLUSTER node allocation"​ window identify the name of the Decomp ​process (middle click on the crystal number) and replace it in the files +
-  * change the n000x to offline and the n000y to online using the variables Decomp???​_CS_Enable +
- +
- +
-====== DAQ cold restart ====== +
-  * Switch off all the vme crates. +
-  * wait 30 s +
-  * Switch on the vme crate of the trigger cards. +
-  * wait till the red lights over the trigger board are on (~ 3 minutes) +
-  * Switch on all the other vme crates +
-  * wait till the green blinking lights appears on all the digitizers (~ 3 minutes) +
-  * start the daq and check the synchronization (alarm window ​or TS window). If few of the digitizers are out of sync recover them using the "​reboot IOCs button",​ otherwise power-cycle on their crate+
  
 +  * Switch to a1
 +    * If a2 cannot be brought back. Gretina DAQ can still run on a1. In the file ~/.profile , environment variable **DAQSERVER** needs changed to be a1.gam . This parameter is used to restrict gretina daq critical scripts to run on specific machine.

Trace:

QR Code
QR Code daq:daq (generated for current page)