Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
daq:daq [2012/07/01 08:02]
weisshaa [Files/trees on GRETINA cluster to go]
daq:daq [2023/11/28 10:47]
stephen
Line 1: Line 1:
-GRETINA DAQ system +======= ​FRIB online build window ​====== 
-====== issues ​======== +Issue the following command prior to starting the ReadoutGUIexport COINCIDENCE_TICKS=300
-ANY issue/​breakdown put here[[:​daqtrouble|daqtrouble]]+
  
-====== start procedures ====== 
  
-  ​[[GRETINA:​GRETINA]] +======= Turn on/off Mode3 Data collection ====== 
-  *[[s800daq:​s800daq]] +  ​To start/stop mode3 data collection on the EPICS GUI page click the button Cluster. In the newly opened window click the button **2 only** for mode2 data only or the **2 and 3** for both mode2 and mode3 data. 
-  *[[source measurement:​source measurement]] +  * Alternatively in a terminal on a2.gam run the command **"​caput Data_CS_Save2And3 0/1"** to turn mode3 collection off/on. 
-  * S800 monitor [[https://​groups.nscl.msu.edu/opdevtech/wiki/index.php/​S800_DAQ_status_tool|S800mon]]+  * After doing this the decomp scripts should be modified as running the syncrofix command could overwrite this.  
 +  * For GRETINA standalone the script to modify is is **grdecompSA.sh** and for GRETINA-S800, **grdecompParticleGamma.sh**. 
 +  * For both scripts change the line **"​caput Data_CS_Save2And3 0/1"** to turn mode3 collection off/on 
 +  
 +======= New in 2022 ====== 
 +start run window is obsoleteTwo scripts are available for changing the data directory. Daq needs to be stopped, directories need to exist:  
 +  * changeDataDir_maintainRunNum ​ <data dir> ​ : keeps the Run number the same, for when we need to move physics experiment data collection to another disk 
 +  * changeDataDir_resetRunNum <data dir> ​ : finds the lowest available Run number without any larger RunDir #s
  
  
-====== Status ====== 
-[[:​status|status]] 
  
 +======= This is DAQ page from previous campaigns ======
 +The DAQ page from the former S800-GRETINA campaign in 2012 can be found at
 +https://​wikihost.nscl.msu.edu/​gretina/​doku.php/​daq:​daq_2012
  
 +  * S800-Gretina control [[docsS800Grt|docsS800Grt]] ​
 +  * {{ :​daq:​ulm-e16011_names.png?​linkonly |S800 Trigger GUI}}
 +======= Troubleshoot ======
 +  * S800-Gretina not correlated: Check CC setfile ~s800/​converged_daq/​Scripts/​CC0105Begin.tcl for extsynch and extclock
 +  * S800 is losing events. Check the network speed of spdaq32 by /​sbin/​ethtool eth0 and /​sbin/​ethtool eth1 , both should show speed as 1000Mb/s, if one shows 100Mb/s then the network is too slow and S800 cannot send data to GEB fast enough which will cause event loss happen.
 +  * Check the ringbuffer status, $DAQBIN/​ringbuffer status - -host=spdaq32 - -all , if the free colum becomes zero, DAQ related people needs to be notified for diagnostics.
  
-==== Files/trees on GRETINA cluster to go ==== +======= ​Decomp Information ​====== 
-(Only to be done by Dirk)+  * [[easy_decomp_calibrations|Easy Gretina Decomp File Creation]] 
 +  * [[decomp_calibrations|Gretina Decomp Calibrations Explanation]]
  
-   /​global/​data1x/​gretina: ​ +======= Gretina Computing ======= 
-   drwxrwxr-x ​ 64 gretina users  4096 Sep 19  2011 CR-1 +Gretina computer farm is maintained by LBNL computer group and local NSCL IT department. Nonetheless I put documents here, but we do NOT do anything to the GRETINA computers!
-   ​drwxrwxr-x 528 gretina users 20480 Oct  4  2011 CR-2 +
-   ​drwxrwxr-x 182 gretina users  8192 Dec  3  2011 CR-4 +
-   ​drwxrwxr-x ​ 12 gretina users  4096 Nov 16  2011 CR-5 +
-   ​drwxrwxr-x 141 gretina users  8192 Aug 24  2011 ER-3 +
-   ​drwxrwxr-x 124 gretina users  4096 May 31  2011 ER-4 +
-   ​drwxrwxr-x 180 gretina users  8192 Jul 11  2011 ER-5 +
-   ​drwxrwxr-x ​ 72 gretina users  4096 Dec 15 12:03 ER-6+
  
-   /​global/​data1b/​gretina 
-   ​2.7T CR-9 
  
-==== File Structure on GRETINA cluster ​files to be kept ====+  * May 2019, documentation provided from Jackie: [[https://​docs.google.com/​document/​d/​1Aq5NkuEKBc4_5fx-95d2RaSZM08VQwVleYWr42QTsFg/​edit?​invite=CNr_nogM&​ts=5cd488d9|Gretina System Documentation]] 
 +  * Instruction to shut down switches for NSCL power shutdown 12/5/2015 {{:​daq:​gretinaswitchesshutdown.pdf|}}
  
-   Here offline decomp data gets stored: +======= NSCL GRETINA LaBr Array =======
-    /​global/​data1a/​gretina/​user1 +
-    /​global/​data1a/​gretina/​user2 +
-    /​global/​data1a/​gretina/​user2_1 +
-    /​global/​data1a/​gretina/​user4+
  
-   Base + data to create them: +    ​* [[fastscint_daq| FastScint DAQ Quick Start]] 
-   /​global/​data1a/​ +    * [[fastscint_daq_long | FastScint DAQ Details]]
-   ​drwxrwxrwx ​ 8 mario   ​bin ​  4096 Mar 27 12:52 q1a1 +
-   ​drwxr-xr-x ​ 6 mario   ​bin ​  4096 Nov 24  2010 q1a3 +
-   ​drwxrwxrwx ​ 9 mario   ​bin ​  4096 Mar 27 12:53 q1b1 +
-   ​drwxr-xr-x ​ 7 mario   ​bin ​  4096 Mar 27 12:53 q1b2 +
-   ​drwxr-xr-x ​ 9 mario   ​bin ​  4096 Mar 27 12:55 q2a5 +
-   ​drwxr-xr-x ​ 9 mario   ​bin ​  4096 Mar 27 12:57 q2a6 +
-   ​drwxrwxrwx ​ 7 mkp     ​bin   4096 Mar 27 13:01 q2b4 +
-   drwxr-xr-x ​ 7 mkp     ​bin ​  4096 Mar 27 13:03 q2b6 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:05 q3a14 +
-   ​drwxr-xr-x ​ 6 mario   ​bin ​  4096 Mar 27 13:06 q3a7 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:08 q4a4 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:09 q4a8 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 13:11 q4b7 +
-   ​drwxr-xr-x ​ 6 mario   ​bin ​  4096 Mar 29 14:00 q4b8 +
-   /​global/​data1b/​ +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 14:26 q3b14 +
-   ​drwxr-xr-x ​ 5 mario   ​bin ​  4096 Mar 27 14:27 q3b5 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  6 18:07 q5a10 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  6 18:17 q5a12 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  7 13:14 q5b10 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar  6 18:27 q5b3 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:29 q6a15 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:29 q6a3 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Jun 10  2011 q6b11 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Jun 10  2011 q6b9 +
-   ​drwxr-xr-x ​ 3 gretina bin   4096 Mar 27 14:31 q7a11 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:32 q7a9 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:33 q7b13 +
-   ​drwxr-xr-x ​ 5 gretina bin   4096 Mar 27 14:33 q7b15 +
-   ​384G DNLapril2012: ​  DNL data of digitizers+
  
-====== ​Configuration files for decompostion ​====== +======= Misc 2019 ====== 
-The configuration for the decomposition is here: https://​wikihost.nscl.msu.edu/​gretina/​doku.php/​daq:​decomp+  * Dual homed NSCL machines are spdaq32 and spdaq07
  
-====== ​Positioning of the quads ====== +====== ​miscellaneous ​====== 
-When a position of a detector ​is changed the following has to be updated: +  * Validation/​Trigger window width is set by Dig116_CS_X55WidthP,​ Dig116_CS_X55WidthN. N/P stands for positive/​negative maximum difference relative ​to Trigger timestamp which will be read out. 
-  * CS_DataId in https://wikihost.nscl.msu.edu/gretina/doku.php/daq:decomp +  * Bringing up just the EPICs screenin gretina home **./startGRETINA_EPICSdisplay** 
-  * CS_GblId in /​global/​devel/​boot/​vmeArray/​vme*.cmd --- The Id is given by  hole_number ​16 + position_in_quad * 4 + board_number +  * Set trigger multiplicity:​ **caput Trig0_CS_FSCThr n** (n=0,1,2,3... for mult 1,​2,​3,​4...) 
-  GTSort_shack/DetNum.dat --- self explanatory+  * See trigger pulse width: **CheckLED.sh** ​  
 +  * Change trigger pulse width: **SETLEDWIN <# of 20ns ticks>​** 
 +  * Check timestamps, report boards that are out-of-sync:​ **CheckTS** 
 +  * Force timestamps to reset, DO NOT USE DURING RUN!!!: **SINGLEIMPSYNC** 
 +  * Warning script: run on a2 as gretina ​in ~/scripts: **./warningMessage4.sh ** 
 +  * Check decomp ​stats  ** recordDecomp.sh | head -n50 ** 
 +  * [[Flash digitizer firmware]] 
 +  * Check digitizer firmware version: on a2 or a1 terminal, type: **DIGSNIDFW** ,all digitizers should have same version firmware 
 +  * [[Replace digitizer with different color ones]] 
 +  * [[Replace fuses of digitizer]] 
 +  * [[No valid decomposed events]] 
 +  * Verify connecedt digitizers, at IOC interface, type **dbior**, it will print all identified digitizers within one bank 
 +  * VME related scripts are located at /​global/​devel/​boot/​vmeArray 
 +  * Check connected PVs, at IOC interface, type "​casr"​ 
 +  * [[ Move detectors to new Hole position ]] 
 + 
 +====== Reboot a2 ====== 
 +  * If a2 is rebooted due to system failure, two scripts need to be brought up 
 +    * from ~gretina: nohup scripts/warningMessage4.sh & 
 +    ​from ~gretina/​scripts/​watcherdir:​ nohup watcher & 
 + The first is the alarm script that talks to users The second ​is responsible for two things: writing record{Counters,​Decomp,​Settings} in the current run directory and cleaning up cluster process issues (usually restarting broken processes) soon after starting or stopping runs. 
 + 
 +  ​* ​Switch to a1 
 +    If a2 cannot be brought back. Gretina DAQ can still run on a1. In the file ~/.profile , environment variable **DAQSERVER** needs changed to be a1.gam . This parameter is used to restrict gretina daq critical scripts to run on specific machine.

QR Code
QR Code daq:daq (generated for current page)