Merge branch 'main' of https://gitea.pkm.physik.tu-darmstadt.de/IPKM/python3-damaris

The previous blit path converted the whole axes bbox (several megapixels
on HiDPI displays) from RGBA to premultiplied ARGB on every motion
event, costing about 84 ms per frame on a maximized window. Per frame
now only the thin strips around the old and new line positions are
erased from a cached full-figure background, redrawn and blitted,
which brings the frame cost below 1 ms.

- BlitSafeCanvas paints blitted regions immediately instead of queueing
  them, which kept regular paints partial and wiped the axis labels
- CrosshairCursor coalesces motion events to 60 fps and erases strips
  by direct numpy copy from the background; matplotlib's restore_region
  with a bbox argument addresses a different coordinate frame and
  restores the wrong area, so it is avoided
- after a full render the old line positions are blitted right away,
  otherwise motion events processed before the pending repaint drew the
  cursor at its new position next to the stale one on screen
- clear_display recreates the crosshair so it survives plot switches

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>

.gitea/workflows/debian-build.yml

@@ -2,30 +2,27 @@ name: Build Debian Packages
 
 on:
   push:
-    branches: [ "main" ]
-  pull_request:
+    tags: [ "v*" ]        # releases are tag-driven now
   workflow_dispatch:
 
 jobs:
   build:
     strategy:
       matrix:
-        target: [debian11, debian12, debian13]
+        target: [debian12, debian13]
         include:
-          - target: debian11
-            codename: bullseye
           - target: debian12
             codename: bookworm
           - target: debian13
             codename: trixie
-    
+
     runs-on: ${{ matrix.target }}
 
     steps:
       - name: Install Node.js and git
         run: |
           apt-get update
-          apt-get -y install nodejs git lsb-release
+          apt-get -y install nodejs git lsb-release jq
 
       - name: Checkout repository
         uses: actions/checkout@v3
@@ -33,20 +30,61 @@ jobs:
           github-server-url: https://gitea.pkm.physik.tu-darmstadt.de
 
       - name: Install build dependencies
-        run: |
-          bash requirements.debian
+        run: bash requirements.debian
 
       - name: Build Package
-        run: |
-          dpkg-buildpackage -us -uc -b
+        run: dpkg-buildpackage -us -uc -b
 
       - name: Collect Artifacts
         run: |
           mkdir -p artifacts
           mv -v ../*.deb artifacts/
 
-      - name: Upload Artifacts
-        uses: actions/upload-artifact@v3
-        with:
-          name: python3-damaris-${{ matrix.codename }}.gz
-          path: artifacts/*.deb
+      - name: Publish release with assets
+        env:
+          GITEA_TOKEN: ${{ secrets.DEB_UPLOAD_TOKEN }}
+          SLUG: ${{ gitea.repository }}          # owner/repo
+          TAG: ${{ gitea.ref_name }}             # e.g. v1.2.3
+          CODENAME: ${{ matrix.codename }}
+        run: |
+          set -euo pipefail
+          API="https://gitea.pkm.physik.tu-darmstadt.de/api/v1"
+          AUTH="Authorization: token ${GITEA_TOKEN}"
+
+          # 1. Find an existing release for this tag, if any
+          release_id=$(curl -s -H "$AUTH" \
+            "$API/repos/$SLUG/releases/tags/$TAG" | jq -r '.id // empty')
+
+          # 2. Create the release if it does not exist yet
+          if [ -z "$release_id" ]; then
+            echo "Creating release $TAG"
+            release_id=$(curl -s -X POST -H "$AUTH" \
+              -H "Content-Type: application/json" \
+              -d "$(jq -n --arg tag "$TAG" \
+                    '{tag_name:$tag, name:$tag, draft:false, prerelease:false}')" \
+              "$API/repos/$SLUG/releases" | jq -r '.id')
+          else
+            echo "Reusing existing release $TAG (id=$release_id)"
+          fi
+          echo "release_id=$release_id"
+
+          # 3. Upload each .deb as an asset
+          for deb in artifacts/*.deb; do
+            asset="${CODENAME}_$(basename "$deb")"   # avoid bookworm/trixie name clash
+            echo "Uploading $asset"
+
+            # delete any existing asset with the same name (idempotent re-runs)
+            existing=$(curl -s -H "$AUTH" \
+              "$API/repos/$SLUG/releases/$release_id/assets" \
+              | jq -r --arg n "$asset" '.[] | select(.name==$n) | .id')
+            if [ -n "$existing" ]; then
+              curl -s -X DELETE -H "$AUTH" \
+                "$API/repos/$SLUG/releases/$release_id/assets/$existing"
+            fi
+
+            curl --fail --show-error --silent \
+              -X POST -H "$AUTH" \
+              -F "attachment=@$deb;filename=$asset" \
+              "$API/repos/$SLUG/releases/$release_id/assets"
+          done
+

.gitignore

@@ -11,3 +11,14 @@ debian/files
 .junie
 debian/debhelper-build-stamp
 debian/.debhelper
+
+# Sphinx documentation
+doc/_build/
+doc/_autosummary/
+doc/damaris/
+
+# Debian build artifacts
+debian/*.debhelper.log
+debian/*.debhelper
+debian/*.substvars
+debian/python3-damaris/

AGENTS.md

@@ -0,0 +1,20 @@
+# python3-damaris
+
+## GUI Purpose and Operation
+This is the front end part of GUI for controlling NMR spectrometers. On "Start", the experiment script executes and creates XML job files. A backend, not in this code, executes these jobs sequentially adn writes the results as XML files. The front end consumes these result files using the result script.
+
+## Guidelines
+1. Prefer the system python and notify the user if libraries and packages are missing. Use uv for the python3 environment only if system packages are missing and the user does not want to install the needed packages.
+2. Old code was ported from gtk2 and python2.7. This code has to work on python 3.11 and above, as well as GTK3
+3. Be aware the targeted Debian versions are bookworm, trixie, and forky.
+4. Only use packages and libraries available in Debian bookworm, trixie, and forky. Make sure that the code is compatible with all of these versions.
+5. Do not commit changes (with git) or write commit messages without asking the user.
+6. Try to minimize code changes, do not use elaborate checks for defensive programming. Ask the user if something should be tested with unit tests.
+7. Use the python unittest module for unit tests, not pytest
+8. Functions for the experiment script belong to the src/damaris/experiments/Experiment.py file
+9. Functions for the result script belong in the files in the src/damaris/data/ folder
+10. The gitea workflow can be found at .gitea/workflows/debian-build.yml
+11. Do not edit anything in "build", it is regenerated
+12. Use sphinx for documentation, autodoc and autosummary is used for references of all classes, functions and modules. 
+13. If a new function/method is created document a short usage in doc/api_reference.rst and a longer explanation in doc/manual.rst
+14. Do not use the Makefile in ./doc/, Doxygen is not used anymore.

DEVELOPER.md

@@ -0,0 +1,45 @@
+# Developing
+
+## Version bump
+
+Version is changed in src/damaris/__init__.py
+
+### Process
+
+1. Update the version in `src/damaris/__init__.py`
+2. Add a new changelog entry in `debian/changelog` using the `dch` (debchange) tool
+
+### Using dch tool
+
+The `dch` tool (from the `devscripts` package) is the standard way to maintain debian changelog files. Example workflow:
+
+```bash
+# Update version in Python package
+sed -i 's/__version__ = "0.24"/__version__ = "0.25"/' src/damaris/__init__.py
+
+# Add changelog entry with dch
+dch --newversion 0.25
+
+# This opens your editor to add the changes for this version
+# Or use batch mode with message:
+dch -v 0.25 "New features and bug fixes"
+
+# Verify the changelog
+tail -20 debian/changelog
+```
+
+The `dch` command will:
+- Create a new entry with the specified version
+- Set the distribution to "stable" (or "unstable" for development versions)
+- Use the maintainer name and email from the previous entry or environment variables
+- Open your editor to add the change descriptions
+
+For development versions, use:
+```bash
+dch -v 0.25-1 --distribution unstable "Development version"
+```
+
+Remember to install devscripts if needed:
+```bash
+sudo apt-get install devscripts
+```

LICENSE

@@ -0,0 +1,338 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+                            NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Moe Ghoul>, 1 April 1989
+  Moe Ghoul, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.

LICENSE.md

@@ -0,0 +1,101 @@
+# GNU General Public License, version 2
+
+**Version 2, June 1991**
+
+Copyright (C) 1989, 1991 Free Software Foundation, Inc.  
+51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
+
+Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
+
+## Preamble
+
+The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too.
+
+When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.
+
+To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.
+
+For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.
+
+We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.
+
+Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations.
+
+Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all.
+
+The precise terms and conditions for copying, distribution and modification follow.
+
+## TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+### Section 0
+This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does.
+
+### Section 1
+You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.
+
+### Section 2
+You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions:
+
+*   **a)** You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change.
+*   **b)** You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.
+*   **c)** If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.
+
+### Section 3
+You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:
+
+*   **a)** Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
+*   **b)** Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
+*   **c)** Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.
+
+If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code.
+
+### Section 4
+You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.
+
+### Section 5
+You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it.
+
+### Section 6
+Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License.
+
+### Section 7
+If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances.
+
+It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice.
+
+This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License.
+
+### Section 8
+If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License.
+
+### Section 9
+The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation.
+
+### Section 10
+If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be led by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally.
+
+## NO WARRANTY
+
+### Section 11
+BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+### Section 12
+IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+**END OF TERMS AND CONDITIONS**

MANIFEST.in

@@ -2,5 +2,6 @@ include src/damaris/gui/DAMARIS3.png
 include src/damaris/gui/DAMARIS3.ico
 include src/damaris/gui/damaris3.glade
 include src/damaris/gui/python.xml
+include scripts/damaris-cli
 #recursive-include doc/reference-html *.html *.gif *.png *.css
 #recursive-include doc/tutorial-html *.html *.gif *.png *.css *.tar.gz *.sh

debian/changelog

@@ -1,3 +1,32 @@
+python3-damaris (0.25) stable; urgency=medium
+
+  * fixed non working stop button
+  * added type hints to data classes, fixed get_ydata override conflict
+  * fixed y_square latent bug: kept always a list of per-channel arrays
+  * fixed time_period latent crashes: missing time information treated as None
+  * replaced remaining gtk2/PyGTK and Python 2 idioms in DamarisGUI
+  * fixed startup crash: register GtkSourceView GType for GtkBuilder
+  * fixed bug with counter updates
+  * FitResult fit overlays, leaked HDF5 files closed after experiment stop
+  * HTML documentation is built at package build time and opened from the Help menu
+  * crosshair cursor is fast on HiDPI screens (dirty-strip blitting)
+
+ -- Markus Rosenstihl <markus.rosenstihl@pkm.tu-darmstadt.de>  Sun, 18 Oct 2026 19:13:20 +0200
+
+python3-damaris (0.24) stable; urgency=medium
+
+  * fixed log dump race condition, no more crash at experiment end
+  * data pool HDF5 file is now always closed properly
+  * Signalpath.phase() no longer modifies data in place, returns object for phase=0 (fixes #43)
+  * fixed inconsistencies in returned objects: missing channels, is_clipped
+  * fixed accumulation lock leaks and deprecated API usage
+  * user plugin directory ~/.config/damaris/plugin added to Python path
+  * all output switched to logging module, experiment and result script prints show up in the GUI
+  * command line interface fixed and properly added to the deb package
+  * performance: incremental writing of HDF5 data pool, reduced polling in synchronize()
+
+ -- Markus Rosenstihl <markus.rosenstihl@pkm.tu-darmstadt.de>  Sun, 18 Oct 2026 11:39:54 +0200
+
 python3-damaris (0.22) stable; urgency=medium
 
   * fixed autocomplete in the editor tabs 

debian/control

@@ -2,7 +2,7 @@ Source: python3-damaris
 Section: science
 Priority: optional
 Maintainer: Markus Rosenstihl <markus.rosenstihl@pkm.tu-darmstadt.de>
-Build-Depends: python3, debhelper, python3-dev, dh-sequence-python3, dpkg-dev, pybuild-plugin-pyproject, sphinx-common, sphinx-rtd-theme-common, python3-sphinxcontrib.bibtex, python3-sphinx-rtd-theme
+Build-Depends: python3, debhelper, python3-dev, dh-sequence-python3, dpkg-dev, pybuild-plugin-pyproject, sphinx-common, python3-sphinx, sphinx-rtd-theme-common, python3-sphinxcontrib.bibtex, python3-sphinx-rtd-theme
 Standards-Version: 3.7.3
 X-Python3-Version: >= 3.5
 Rules-Requires-Root: no
@@ -10,7 +10,7 @@ Rules-Requires-Root: no
 Package: python3-damaris
 Architecture: all
 Provides: damaris-frontend-py3
-Depends: ${python3:Depends}, python3-numpy, python3-scipy, python3-gi, python3-matplotlib, python3-tables, lzop, python3-xdg, libgtksourceview-3.0-1, libgirepository1.0-dev | libgirepository-2.0-dev, breeze-icon-theme
+Depends: ${python3:Depends}, ${sphinxdoc:Depends}, python3-numpy, python3-scipy, python3-gi, python3-matplotlib, python3-tables, lzop, python3-xdg, libgtksourceview-3.0-1, libgirepository1.0-dev | libgirepository-2.0-dev, breeze-icon-theme
 Recommends: damaris-backends, python3-numpy-ext
 Suggests: python3-doc, python3-tables-doc, python3-numpy-doc
 Description: python frontend for DAMARIS project

debian/install

@@ -1,4 +1,5 @@
-doc/ usr/share/doc/python3-damaris/
+build/html usr/share/doc/python3-damaris/
 debian/damaris3.desktop usr/share/applications/
 src/damaris/gui/DAMARIS3.png usr/share/pixmaps/
 src/damaris/gui/DAMARIS3.ico usr/share/pixmaps/
+scripts/damaris-cli usr/bin/

debian/rules

@@ -17,7 +17,7 @@ DH_ALWAYS_EXCLUDE=CVS:.git:venv:.venv:*.h5
 
 override_dh_auto_build:
 	dh_auto_build
-	sphinx-build -N -bhtml doc/ build/doc
+	sphinx-build -N -bhtml doc/ build/html
 
 
 override_dh_auto_test:

doc/api_reference.rst

@@ -0,0 +1,107 @@
+#############
+API Reference
+#############
+
+This page provides an auto-generated reference of all DAMARIS classes, functions, and modules.
+It is built from docstrings using Sphinx autodoc and autosummary.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Modules:
+
+   damaris/data
+   damaris/experiments
+   damaris/gui
+   damaris/tools
+
+
+Data Module
+===========
+
+The data module contains result types, data handling, and signal processing classes.
+
+.. autosummary::
+   :toctree: _autosummary
+   :recursive:
+
+   damaris.data.ADC_Result
+   damaris.data.Accumulation
+   damaris.data.Config_Result
+   damaris.data.DamarisFFT
+   damaris.data.DataPool
+   damaris.data.Drawable
+   damaris.data.Errorable
+   damaris.data.Error_Result
+   damaris.data.FitResult
+   damaris.data.MeasurementResult
+   damaris.data.Persistance
+   damaris.data.Resultable
+   damaris.data.Signalpath
+   damaris.data.Temperature
+   damaris.data.autophase
+
+
+Experiments Module
+==================
+
+The experiments module contains the Experiment class and related utilities for defining pulse sequences.
+
+.. autosummary::
+   :toctree: _autosummary
+   :recursive:
+
+   damaris.experiments.Experiment
+   damaris.experiments.DurationEstimator
+   damaris.experiments.dac
+
+
+GUI Module
+==========
+
+The GUI module contains the graphical interface, backend driver, and experiment/result handling.
+
+.. autosummary::
+   :toctree: _autosummary
+   :recursive:
+
+   damaris.gui.BackendDriver
+   damaris.gui.DamarisGUI
+   damaris.gui.ExperimentHandling
+   damaris.gui.ExperimentWriter
+   damaris.gui.ResultHandling
+   damaris.gui.ResultReader
+   damaris.gui.TemperatureMonitor
+   damaris.gui.logging_utils
+
+A short usage example for the font helper of the GUI module (the non-deprecated
+replacement for the removed ``Widget.modify_font()``):
+
+.. code-block:: python
+
+   from damaris.gui.DamarisGUI import apply_font_css
+
+   apply_font_css(textview, "Monospace 10")   # pango style font description
+   apply_font_css(label, "sans bold 16")
+
+The font string is converted into a CSS rule (``font-family``/``font-size``/
+``font-weight``) which is applied to the widget's style context with a
+``Gtk.CssProvider``.
+
+
+Tools Module
+============
+
+The tools module contains hardware driver interfaces and utility functions.
+
+.. autosummary::
+   :toctree: _autosummary
+   :recursive:
+
+   damaris.tools.eurotherm
+   damaris.tools.flow_control
+   damaris.tools.goniometer
+   damaris.tools.lakeshore
+   damaris.tools.ranges
+   damaris.tools.signal
+   damaris.tools.temperature
+   damaris.tools.temperature_simulator

doc/conf.py

@@ -21,9 +21,13 @@ release = damaris.__version__
 
 extensions = [
     'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
     'sphinx.ext.napoleon',
+    'sphinx.ext.viewcode',
     'sphinxcontrib.bibtex',
     ]
+
+autosummary_generate = True
 bibtex_bibfiles = ['bibdesk_db.bib']
 bibtex_bibstyles = ['numeric']
 bibtex_citestyle = 'numeric'

doc/devel.rst

@@ -184,3 +184,57 @@ Stability of the DAC has been tested by measuring the shifts of resonance freque
 
 The offset of the PFG amplifier was adjusted via the offset potentiometer on the DAC board. There is also an offset adjustment possible at the PFG amplifier directly. For adjustment, the signal of water in a sample tube with a 0.5 mm teflon stripe perpendicular to the magnetic field B0 was recorded. First, the PFG power supply has been switched off and the sample was shimmed with the room temperature shims until the longest decay time T*2 of the FID has been achieved. After this, the PFG was switched on and the offset has been adjusted to cause least distortion in the spectrum.
 
+*******************
+Publish new version
+*******************
+
+.. warning::
+
+   ``__version__`` in ``src/damaris/__init__.py`` and the top entry of
+   ``debian/changelog`` must carry the same version number at every
+   release. If they drift apart, the Debian package version no longer
+   matches the application version.
+
+1. Commit all code changes intended for the release.
+
+2. Bump ``__version__`` in ``src/damaris/__init__.py``. This constant
+   is the single source of truth for the application version and
+   follows the simple ``X.Y`` scheme used so far (a minor release
+   increments the second number, e.g. 0.22 -> 0.23).
+
+3. Add a new entry at the top of ``debian/changelog`` with the same
+   version number and a bullet list of the user visible changes. The
+   Debian package version does not follow ``__version__``
+   automatically; it is defined solely by this file. Either edit
+   ``debian/changelog`` directly or run ``dch`` (from the
+   ``devscripts`` package), which prepends the entry for you. Match
+   the entry format already used in this file:
+
+   .. code-block:: text
+
+      python3-damaris (X.Y) stable; urgency=medium
+
+        * bullet list of user visible changes
+
+       -- Full Name <email>  Day, DD Mon YYYY HH:MM:SS +0200
+
+   The maintainer line at the end is indented by two spaces and its
+   date must be in RFC-2822 format, which ``date -R`` produces.
+
+4. Commit the version bump and the changelog entry together in one
+   commit. The repository currently carries no version tags, so no
+   tagging step is defined; confirm with ``git tag -l`` before
+   introducing one.
+
+5. Everything else picks up the new version automatically. The Sphinx
+   documentation reads ``damaris.__version__`` in ``doc/conf.py``, and
+   ``pyproject.toml`` declares the package version dynamically via
+   ``[tool.setuptools.dynamic]`` with ``attr = "damaris.__version__"``,
+   so neither needs a manual edit. On a push to ``main`` the Debian CI
+   pipeline ``.gitea/workflows/debian-build.yml`` builds the package
+   with ``dpkg-buildpackage`` for bookworm and trixie; the version of
+   the resulting ``.deb`` is taken from ``debian/changelog``.
+
+``DEVELOPER.md`` at the repository root keeps a short note about the
+version bump in ``src/damaris/__init__.py``; this section expands it.
+

doc/manual.rst

@@ -101,6 +101,12 @@ There are now two possibilities to export the figure:
 2. Saving the data of the plot as CSV file (comma separated values) by pressing the button on the right side of the toolbar under the plot
 
 
+Help Menu
+=========
+
+The **Help** button in the toolbar opens this documentation in the system's web browser. The arrow next to it offers additional local documentations, such as the Python documentation, if installed. When the Debian package is used, the HTML documentation is found in ``/usr/share/doc/python3-damaris/html/``.
+
+
 Accumulation
 ============
 
@@ -285,6 +291,35 @@ A typical method to already extract data like amplitudes etc. is possible by usi
 
 Essentially, a dictionary "measurement" is created with *tau* as the key. For each new key the current magnetization is added. This dictionary is then added to the data pool. The advantage over the previous listing is that the magnetization vs. τ curve can be directly displayed in order to gain an overview of the running experiment, because the "measurement" dictionary is added to the data pool. In this example, the value also stores statistics from the accumulation. The data dictionary is stored automatically as described before, including the result and experiment scripts, information about the used back end and a timeline of the experiment.
 
+**Displaying fits**
+
+A fit to a measurement can be shown as an overlay on the measurement curve using the FitResult class. In contrast to the measurement itself, a fit has its own x-values, independent of the measured data points. Therefore the fit can be sampled on a much denser x-grid, which makes the plotted curve smooth. The fit is stored in the data pool under its own key and is drawn by the GUI on top of the measurement it belongs to. Fit parameters are shown in the plot and are stored together with the fit.
+
+The fit is associated with its measurement in two ways: by the ``source`` argument (the data pool key of the measurement) and, optionally, by storing it under a nested key such as ``"Magnetization(tau)/fit"``, so that it is stored next to the measurement in the HDF5 data pool file. The GUI shows the fit only as an overlay on its measurement; fits do not get their own entry in the display tree.
+
+.. code-block:: python
+   :linenos:
+
+   import numpy
+   from damaris import FitResult
+
+   def result(results):
+       # ... fill "measurement" as in the listing above ...
+       # fit an exponential decay to the measured points
+       from scipy.optimize import curve_fit
+       [k, v, e] = measurement.get_errorplotdata()
+       popt, pcov = curve_fit(lambda t, m0, t1: m0 * numpy.exp(-t / t1),
+                              k, v, p0=(v[0], 1.0), sigma=e)
+       # sample the fit on a dense x-grid for a smooth curve
+       xfit = numpy.linspace(k.min(), k.max(), 200)
+       fit = FitResult(xfit, popt[0] * numpy.exp(-xfit / popt[1]),
+                       title="Magnetization(tau) fit",
+                       source="Magnetization(tau)",
+                       params={"M0": popt[0], "T1": popt[1]})
+       data["Magnetization(tau)/fit"] = fit
+
+The fit is written to the HDF5 data pool file as its own table (columns *x* and *y*, plus *y_err* if given), together with the fit parameters as attributes, and is restored when the data pool is read back.
+
 Note that we can also create a HDF5 file from scratch! In this case experiment and result script are not saved in the resulting file. The advantage is that one can create arbitrary complex structured files at the cost of having a more complicated result script.
 
 .. code-block:: python
@@ -313,6 +348,10 @@ The change is *in-place*, which means if the timesignal is needed later on, a co
 - **clip(start=**\ *None* **, stop=**\ *None* **)**
    Only the data between start and stop is returned.
    The start and stop parameters can be either in time or frequency domain.
+   Calling **clip()** without any argument is a placeholder that leaves the
+   object unchanged (useful in scripted processing chains). If only one bound
+   is given, the missing one defaults to the first resp. last data point and
+   reversed bounds (start > stop) are swapped automatically.
 
 - **baseline(lastpart=**\ *0.1* **)**
    This method should be applied first as it corrects the baseline of the signal, taking the last part of the signal for determining the correction.
@@ -374,6 +413,12 @@ To use this methods one has to apply consecutively the methods to an *timesignal
 This applies baseline correction using the last 40% of the timesignal for determination of the signal. Then, an exponential window is applied. From this timesignal an FFT is calculated which is further clipped to view to the data between -1 kHz with 1 kHz width.
 
 
+Operators and Copies of Data Objects
+====================================
+
+The arithmetic operators (``+``, ``-``, ``*``, ``/``, ``//``, ``**``, unary ``-``) of **ADC_Result** and **Accumulation** objects return *new* objects: the operands are never modified and the data arrays of the result are independent copies, so later in-place operations (``+=``, ``*=``, ...) on the result cannot corrupt the operands. The channel metadata is carried over to the result: an **ADC_Result** result keeps the same ``get_nChannels()`` and ``is_clipped`` flag as its operand, and an **Accumulation** result keeps (and merges) the ``job_ids`` provenance of both operands. Adding an unsupported type or negating an empty accumulation raises a ``ValueError`` instead of silently returning *None*. In the same spirit, ``Config_Result(config=..., desc=...)`` stores the given dictionaries in the constructor, and ``measurementresult + 0`` returns an independent deep copy of the result object.
+
+
 Good Practices
 ==============
 

pyproject-bookworm.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "python3-damaris"
+version = "0.25"
+requires-python = ">= 3.11"
+description = "Python 3 frontend for DAMARIS (DArmstadt MAgnetic Resonance Instrument Software)"
+license = {text = "GPL-2.0"}
+keywords = ["NMR", "data-processing"]
+readme = {file = "README", content-type = "text/plain"}
+dependencies = [
+    "scipy>=1.7",
+    "numpy>=1.21,<2.0",
+    "tables>=3.7",
+    "matplotlib>=3.5,<3.11",
+    "pyxdg",
+    # PyGObject must come from system packages:
+    #   apt-get install python3-gi gir1.2-gtk-3.0 gir1.2-gtksource-3.0
+]
+authors = [
+  {name = "Achim Gädke"},
+  {name = "Christopher Schmitt"},
+  {name = "Christian Tacke"},
+  {name = "Joachim Beerwerth"},
+  {name = "Markus Rosenstihl", email = "markus.rosenstihl@pkm.tu-darmstadt.de"}
+]
+
+[project.scripts]
+DAMARIS3 = "damaris.__main__:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+"damaris.gui" = ["DAMARIS3.png", "DAMARIS3.ico", "python.xml", "damaris3.glade"]

pyproject.toml

@@ -6,7 +6,7 @@ requires = [
 build-backend = "setuptools.build_meta"
 
 [project]
-requires-python = ">= 3.9"
+requires-python = ">= 3.11"
 name = "python3-damaris"
 dynamic = ["version"]
 description = "Python version 3 frontend for DAMARIS (DArmstadt MAgnetic Resonance Instrument Software)"
@@ -21,6 +21,9 @@ dependencies = [
     "pyxdg",
     "PyGObject >= 3.14.0, <= 3.51.0",
     "pyserial>=3.5",
+    "sphinx>=9.0.4",
+    "sphinxcontrib-bibtex>=2.7.0",
+    "sphinx-rtd-theme>=3.1.0",
 ]
 authors = [
   {name = "Achim Gädke"},
@@ -36,6 +39,9 @@ maintainers = [
 [project.scripts]
 DAMARIS3 = "damaris.__main__:main"
 
+[dependency-groups]
+dev = ["pyrefly"]
+
 [tool.setuptools.packages.find]
 where = ["src"]
 
@@ -47,6 +53,18 @@ version = {attr = "damaris.__version__"}
 
 [tool.setuptools.data-files]
 "share/python3-damaris/images" = ["src/damaris/gui/DAMARIS3.png", "src/damaris/gui/DAMARIS3.ico"]
+
+[tool.pyrefly]
+project-includes = [
+    "src/damaris/experiments/Experiment.py",
+    "src/damaris/experiments/dac.py",
+    "src/damaris/data/ADC_Result.py",
+    "src/damaris/data/MeasurementResult.py",
+    "src/damaris/data/Accumulation.py",
+]
+search-path = ["src"]
+python-version = "3.11"
+python-platform = "linux"
 #"share/python3-damaris/doc" = ["doc/index.html"]
 #"share/python3-damaris/doc/reference-html" = ["doc/reference-html/*"]
 #"share/python3-damaris/doc/tutorial-html" = ["doc/tutorial-html/*"]

requirements.debian

@@ -1,7 +1,7 @@
 #!/bin/bash
 apt-get -y install libcairo2-dev gir1.2-gtksource-3.0 hdf5-tools
 apt-get -y install python3-dev python3-scipy python3-lmfit python3-matplotlib python3-tables python3-xdg  python3-setuptools python3-sphinx-rtd-theme
-apt-get -y install debhelper dh-sequence-python3 lsb-release sphinx-common pybuild-plugin-pyproject python3-sphinxcontrib.bibtex
+apt-get -y install debhelper dh-sequence-python3 lsb-release sphinx-common python3-sphinx pybuild-plugin-pyproject python3-sphinxcontrib.bibtex
 DEBRELEASE=$(lsb_release -cs)
 case $DEBRELEASE in
   "trixie")

scripts/damaris-cli

@@ -1,9 +1,11 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 
 import time
 import sys
 import os
 import os.path
+import argparse
+import logging
 import tables
 import damaris.data.DataPool as DataPool
 import damaris.gui.ResultReader as ResultReader
@@ -12,7 +14,9 @@ import damaris.gui.BackendDriver as BackendDriver
 import damaris.gui.ResultHandling as ResultHandling
 import damaris.gui.ExperimentHandling as ExperimentHandling
 
-def some_listener(event):
+logger = logging.getLogger("damaris.cli")
+
+def progressbar(event):
     if event.subject=="__recentexperiment" or event.subject=="__recentresult":
         r=event.origin.get("__recentresult",-1)+1
         e=event.origin.get("__recentexperiment",-1)+1
@@ -20,17 +24,19 @@ def some_listener(event):
             ratio=100.0*r/e
         else:
             ratio=100.0
-        print("\r%d/%d (%.0f%%)"%(r,e,ratio), end=' ')
+        sys.stderr.write("\r%d/%d (%.0f%%)"%(r,e,ratio))
+        sys.stderr.flush()
 
 
 class ScriptInterface:
 
-    def __init__(self, exp_script=None, res_script=None, backend_executable=None, spool_dir="spool"):
+    def __init__(self, exp_script=None, res_script=None, backend_executable=None, spool_dir="spool", log_callback=None):
         self.exp_script=exp_script
         self.res_script=res_script
         self.backend_executable=backend_executable
         self.spool_dir=os.path.abspath(spool_dir)
         self.exp_handling=self.res_handling=None
+        self._log_callback = log_callback
 
         self.exp_writer=self.res_reader=self.back_driver=None
         if self.backend_executable is not None:
@@ -48,9 +54,9 @@ class ScriptInterface:
     def runScripts(self):
         # get script engines
         if self.exp_script and self.exp_writer:
-            self.exp_handling=ExperimentHandling.ExperimentHandling(self.exp_script, self.exp_writer, self.data)
+            self.exp_handling=ExperimentHandling.ExperimentHandling(self.exp_script, self.exp_writer, self.data, self._log_callback)
         if self.res_script and self.res_reader:
-            self.res_handling=ResultHandling.ResultHandling(self.res_script, self.res_reader, self.data)
+            self.res_handling=ResultHandling.ResultHandling(self.res_script, self.res_reader, self.data, self._log_callback)
 
         # start them
         if self.exp_handling: self.exp_handling.start()
@@ -73,27 +79,29 @@ class ScriptInterface:
                     if not self.exp_handling.is_alive():
                         self.exp_handling.join()
                         if self.exp_handling.raised_exception:
-                            print(": experiment script failed at line %d (function %s): %s"%(self.exp_handling.location[0],
-                                                                                             self.exp_handling.location[1],
-                                                                                             self.exp_handling.raised_exception))
+                            logger.error(": experiment script failed at line %d (function %s): %s",
+                                        self.exp_handling.location[0],
+                                        self.exp_handling.location[1],
+                                        self.exp_handling.raised_exception)
                         else:
-                            print(": experiment script finished")
+                            logger.info(": experiment script finished")
                         self.exp_handling = None
 
                 if self.res_handling is not None:
                     if not self.res_handling.is_alive():
                         self.res_handling.join()
                         if self.res_handling.raised_exception:
-                            print(": result script failed at line %d (function %s): %s"%(self.res_handling.location[0],
-                                                                                         self.res_handling.location[1],
-                                                                                         self.res_handling.raised_exception))
+                            logger.error(": result script failed at line %d (function %s): %s",
+                                        self.res_handling.location[0],
+                                        self.res_handling.location[1],
+                                        self.res_handling.raised_exception)
                         else:
-                            print(": result script finished")
+                            logger.info(": result script finished")
                         self.res_handling = None
 
                 if self.back_driver is not None:
                     if not self.back_driver.is_alive():
-                        print(": backend finished")
+                        logger.info(": backend finished")
                         self.back_driver=None
 
         except KeyboardInterrupt:
@@ -120,33 +128,45 @@ class ScriptInterface:
             dump_file=None
             # todo
         except Exception as e:
-            print("dump failed", e)
+            logger.error("Dump failed: %s", e)
        
 
 
 if __name__=="__main__":
+    parser = argparse.ArgumentParser(
+        description="Run DAMARIS experiment and result scripts."
+    )
+    parser.add_argument("experiment_script", help="Path to the experiment script file")
+    parser.add_argument("result_script", help="Path to the result script file")
+    parser.add_argument(
+        "spool_dir", nargs="?", default=None,
+        help="Spool directory (defaults to current working directory)"
+    )
+    parser.add_argument(
+        "--backend", default="/usr/lib/damaris/backends/Mobilecore",
+        help="Path to the backend executable"
+    )
+    parser.add_argument(
+        "--output", default="data_pool.h5",
+        help="Output HDF5 file path (default: data_pool.h5)"
+    )
+    args = parser.parse_args()
 
-    if len(sys.argv)==1:
-        print("%s: data_handling_script [spool directory]"%sys.argv[0])
-        sys.exit(1)
-    if len(sys.argv)==3:
-        spool_dir=os.getcwd()
-    else:
-        spool_dir=sys.argv[3]
-    
-    expscriptfile=open(sys.argv[1])
-    expscript=expscriptfile.read()
-    resscriptfile=open(sys.argv[2])
-    resscript=resscriptfile.read()
+    spool_dir = args.spool_dir if args.spool_dir else os.getcwd()
 
-    si=ScriptInterface(expscript, resscript,"/usr/lib/damaris/backends/Mobilecore", spool_dir)
+    with open(args.experiment_script) as f:
+        expscript = f.read()
+    with open(args.result_script) as f:
+        resscript = f.read()
 
-    si.data.register_listener(some_listener)
+    si = ScriptInterface(expscript, resscript, args.backend, spool_dir)
+
+    si.data.register_listener(progressbar)
 
     si.runScripts()
 
     si.waitForScriptsEnding()
 
-    si.dump_data("data_pool.h5")
+    si.dump_data(args.output)
 
-    si=None
+    si = None

src/damaris/__init__.py

@@ -15,5 +15,13 @@
 
 ## module contents
 #
-__version__ = "0.22"
+__version__ = "0.25"
 __all__=["experiments", "data", "gui"]
+
+import sys
+import os
+
+# Add user plugin directory to Python path (highest priority)
+_plugin_dir = os.path.expanduser("~/.config/damaris/plugin")
+if os.path.isdir(_plugin_dir) and _plugin_dir not in sys.path:
+    sys.path.insert(0, _plugin_dir)

src/damaris/backend/logging_helper.h

@@ -0,0 +1,134 @@
+// -*- C++ -*-
+/**
+ * @file logging_helper.h
+ * @brief Lightweight logging helper for the DAMARIS C++ backend.
+ *
+ * Writes structured log messages to stdout in a format parseable by the
+ * Python FileTailerHandler:
+ *
+ *   [LEVEL] message
+ *
+ * Levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
+ *
+ * Usage in C++ code:
+ *   #include "logging_helper.h"
+ *   log_info("Starting backend, spool=%s", spool_dir.c_str());
+ *   log_error("Failed to open device: %s", strerror(errno));
+ */
+
+#ifndef DAMARIS_LOGGING_HELPER_H
+#define DAMARIS_LOGGING_HELPER_H
+
+#include <cstdio>
+#include <cstdarg>
+#include <cstring>
+#include <ctime>
+#include <string>
+
+namespace damaris {
+namespace logging {
+
+// ---------------------------------------------------------------------------
+// Internal helpers
+// ---------------------------------------------------------------------------
+
+inline std::string timestamp() {
+    char buf[32];
+    time_t now = time(nullptr);
+    struct tm tm_buf;
+#ifdef _WIN32
+    localtime_s(&tm_buf, &now);
+#else
+    localtime_r(&now, &tm_buf);
+#endif
+    strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S", &tm_buf);
+    return std::string(buf);
+}
+
+inline void log_message(int level, const char* fmt, ...) {
+    const char* level_str = "INFO";
+    switch (level) {
+        case 0: level_str = "DEBUG"; break;
+        case 1: level_str = "INFO";  break;
+        case 2: level_str = "WARNING"; break;
+        case 3: level_str = "ERROR";   break;
+        case 4: level_str = "CRITICAL"; break;
+    }
+
+    char msg[4096];
+    va_list args;
+    va_start(args, fmt);
+    vsnprintf(msg, sizeof(msg), fmt, args);
+    va_end(args);
+
+    fprintf(stdout, "[%s] %s\n", level_str, msg);
+    fflush(stdout);
+}
+
+// ---------------------------------------------------------------------------
+// Convenience macros / inline functions
+// ---------------------------------------------------------------------------
+
+inline void log_debug(const char* fmt, ...) {
+    char msg[4096];
+    va_list args;
+    va_start(args, fmt);
+    vsnprintf(msg, sizeof(msg), fmt, args);
+    va_end(args);
+    fprintf(stdout, "[DEBUG] %s\n", msg);
+    fflush(stdout);
+}
+
+inline void log_info(const char* fmt, ...) {
+    char msg[4096];
+    va_list args;
+    va_start(args, fmt);
+    vsnprintf(msg, sizeof(msg), fmt, args);
+    va_end(args);
+    fprintf(stdout, "[INFO] %s\n", msg);
+    fflush(stdout);
+}
+
+inline void log_warning(const char* fmt, ...) {
+    char msg[4096];
+    va_list args;
+    va_start(args, fmt);
+    vsnprintf(msg, sizeof(msg), fmt, args);
+    va_end(args);
+    fprintf(stdout, "[WARNING] %s\n", msg);
+    fflush(stdout);
+}
+
+inline void log_error(const char* fmt, ...) {
+    char msg[4096];
+    va_list args;
+    va_start(args, fmt);
+    vsnprintf(msg, sizeof(msg), fmt, args);
+    va_end(args);
+    fprintf(stdout, "[ERROR] %s\n", msg);
+    fflush(stdout);
+}
+
+inline void log_critical(const char* fmt, ...) {
+    char msg[4096];
+    va_list args;
+    va_start(args, fmt);
+    vsnprintf(msg, sizeof(msg), fmt, args);
+    va_end(args);
+    fprintf(stdout, "[CRITICAL] %s\n", msg);
+    fflush(stdout);
+}
+
+} // namespace logging
+} // namespace damaris
+
+// ---------------------------------------------------------------------------
+// Optional C-style macros for convenience
+// ---------------------------------------------------------------------------
+#define DAMARIS_LOG_DEBUG(fmt, ...)   damaris::logging::log_debug(fmt, ##__VA_ARGS__)
+#define DAMARIS_LOG_INFO(fmt, ...)    damaris::logging::log_info(fmt, ##__VA_ARGS__)
+#define DAMARIS_LOG_WARNING(fmt, ...) damaris::logging::log_warning(fmt, ##__VA_ARGS__)
+#define DAMARIS_LOG_ERROR(fmt, ...)   damaris::logging::log_error(fmt, ##__VA_ARGS__)
+#define DAMARIS_LOG_CRITICAL(fmt, ...) damaris::logging::log_critical(fmt, ##__VA_ARGS__)
+
+#endif // DAMARIS_LOGGING_HELPER_H

src/damaris/data/ADC_Result.py

@@ -1,5 +1,7 @@
 # -*- coding: iso-8859-1 -*-
 
+from __future__ import annotations
+
 from .Resultable import Resultable
 from .Drawable import Drawable
 from .Signalpath import Signalpath
@@ -9,6 +11,13 @@ import numpy
 import sys
 import datetime
 import tables
+from typing import IO
+from scipy.signal import filtfilt, remez
+
+_XArray = numpy.ndarray | list
+_YChannels = list[numpy.ndarray]
+_IndexList = list[tuple[int, int]]
+_Description = dict[str, str]
 #############################################################################
 #                                                                           #
 # Name: Class ADC_Result                                                    #
@@ -38,7 +47,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
         sampling_rate: Sampling frequency of the ADC data.
         job_id: identifier for the job that generated this result.
     """
-    def __init__(self, x = None, y:list = None, index = None, sampl_freq = None, desc = None, job_id = None, job_date = None):
+    def __init__(self, x: _XArray | None = None, y: _YChannels | None = None, index: _IndexList | None = None, sampl_freq: float | None = None, desc: _Description | None = None, job_id: int | None = None, job_date: datetime.datetime | None = None):
         Resultable.__init__(self)
         Drawable.__init__(self)
 
@@ -58,7 +67,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
             self.cont_data = False
             self.sampling_rate = 0
             self.index = []
-            self.x = []
+            self.x = numpy.array([], dtype="float32")
             self.y = []
         # using all arguments for initialization
         elif (x is not None) and (y is not None) and (index is not None) and (sampl_freq is not None) and (desc is not None) and (job_id is not None) and (job_date is not None):
@@ -66,8 +75,8 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
             #for ch in y:
             #    if not numpy.issubdtype(ch.dtype, numpy.integer):
             #        raise TypeError("TypeError: ADC_Result y data must be a list with integer type channels")
-            self.x = x
-            self.y = y
+            self.x = numpy.asarray(x)
+            self.y = [numpy.asarray(ch) for ch in y]
             self.index = index
             self.sampling_rate = sampl_freq
             self.cont_data = True
@@ -86,30 +95,29 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
             raise ValueError("Wrong usage of __init__!")
 
 
-    def create_data_space(self, channels, samples):
+    def create_data_space(self, channels: int, samples: int) -> None:
         "Initialises the internal data-structures"
 
         if self.contains_data():
             print("Warning ADC-Result: Tried to run \"create_data_space()\" more than once.")
             return
 
-            raise ValueError("ValueError: You cant create an ADC-Result with less than 1 sample!")
-
         for i in range(channels):
             self.y.append(numpy.zeros((samples,), dtype="int16"))
 
         self.x = numpy.zeros((samples,), dtype="float32")
+        self.nChannels = channels
 
         self.index.append((0, samples-1))
         self.cont_data = True
 
 
-    def contains_data(self):
+    def contains_data(self) -> bool:
         """Returns true if ADC_Result contains data. (-> create_data_space() was called)"""
         return self.cont_data
 
 
-    def check_clipping(self, bit_depth=None):
+    def check_clipping(self, bit_depth: int | None = None) -> bool:
         """
         Check if any data in y is clipped.
         For a 14-bit ADC, clipping occurs at 2**13 (8192) or 2**13-1 (8191).
@@ -147,7 +155,99 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
         return self.is_clipped
 
 
-    def add_sample_space(self, samples):
+    def lowpass(self, cutoff: float, numtaps: int | None = None) -> ADC_Result:
+        """
+        Apply a zero-phase lowpass FIR filter to all channels.
+
+        Uses scipy.signal.filtfilt (forward-backward filtering) with a Parks-McClellan
+        FIR filter (remez). The forward-backward pass guarantees zero phase distortion
+        and doubles the effective filter order.
+
+        The stopband is automatically set to 0.1 * Nyquist beyond the cutoff. If
+        numtaps is not provided, it is estimated from the transition width to yield
+        approximately 40-50 dB stopband attenuation (typically ~40 taps).
+
+        Parameters
+        ----------
+        cutoff : float
+            Passband edge frequency in Hz. Frequencies above this value are attenuated.
+            Must be a positive number below the Nyquist frequency.
+        numtaps : int, optional
+            Number of FIR taps. If omitted, computed automatically from the transition
+            width.
+
+        Returns
+        -------
+        ADC_Result
+            A new ADC_Result with the filtered data (float64 per channel).
+            The original object is not modified.
+
+        Raises
+        ------
+        RuntimeError
+            If the result contains no data.
+        ValueError
+            If cutoff is invalid for the current sampling rate.
+
+        Examples
+        --------
+        >>> # Lowpass at 1 MHz (taps auto-computed, ~40)
+        ... filtered = adc.lowpass(cutoff=1e6)
+        >>> # Lowpass at 500 kHz with fixed 63 taps
+        ... filtered = adc.lowpass(cutoff=500e3, numtaps=63)
+        """
+        if not self.contains_data():
+            raise RuntimeError("lowpass: no data present")
+
+        nyquist = self.sampling_rate / 2.0
+        cutoff = float(cutoff)
+
+        if cutoff <= 0 or cutoff >= nyquist:
+            raise ValueError(
+                f"cutoff {cutoff} Hz is outside valid range (0, {nyquist})"
+            )
+
+        # Transition width: 0.1 * Nyquist
+        trans_width = 0.1 * nyquist
+        stopband = min(cutoff + trans_width, nyquist * 0.99)
+
+        # Auto-estimate taps if not given
+        if numtaps is None:
+            numtaps = max(21, int(round(8.0 / (stopband - cutoff) * nyquist)))
+        if numtaps % 2 == 0:
+            numtaps += 1
+
+        coeffs = remez(numtaps, [0, cutoff, stopband, nyquist], [1, 0], fs=self.sampling_rate)
+
+        self.lock.acquire()
+        try:
+            tmp_y = []
+            for i in range(self.get_number_of_channels()):
+                data = numpy.asarray(self.y[i], dtype='float64')
+                filtered = filtfilt(
+                    coeffs, [1.0], data,
+                    padtype='odd',
+                    padlen=(numtaps - 1) * 3
+                )
+                tmp_y.append(filtered)
+
+            r = ADC_Result(
+                x=self.x[:],
+                y=tmp_y,
+                index=self.index[:],
+                sampl_freq=self.sampling_rate,
+                desc=self.description,
+                job_id=self.job_id,
+                job_date=self.job_date,
+            )
+            self._copy_metadata(r)
+        finally:
+            self.lock.release()
+
+        return r
+
+
+    def add_sample_space(self, samples: int) -> None:
         "Adds space for n samples, where n can also be negative (deletes space). New space is filled up with \"0\""
 
         self.lock.acquire()
@@ -167,7 +267,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
         self.lock.release()
 
 
-    def get_result_by_index(self, index):
+    def get_result_by_index(self, index: int) -> ADC_Result:
 
         self.lock.acquire()
         try:
@@ -184,97 +284,97 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
             tmp_y.append(self.y[i][start:end+1].copy())
 
         r = ADC_Result(x = tmp_x, y = tmp_y,  index = [(0,len(tmp_y[0])-1)], sampl_freq = self.sampling_rate, desc = self.description.copy() if self.description is not None else None, job_id = self.job_id, job_date = self.job_date)
+        self._copy_metadata(r)
         self.lock.release()
         return r
 
 
-    def get_sampling_rate(self):
+    def get_sampling_rate(self) -> float:
         """Returns the samplingfrequency"""
         return self.sampling_rate + 0
 
 
-    def set_sampling_rate(self, hz):
+    def set_sampling_rate(self, hz: float) -> None:
         """Sets the samplingfrequency in hz"""
         self.sampling_rate = float(hz)
 
 
-    def get_nChannels(self):
+    def get_nChannels(self) -> int:
         """Gets the number of channels"""
         return self.nChannels + 0
 
-    def set_nChannels(self, channels):
+    def set_nChannels(self, channels: int) -> None:
         """Sets the number of channels"""
         self.nChannels = int(channels)
 
 
-    def get_index_bounds(self, index):
+    def get_index_bounds(self, index: int) -> tuple[int, int]:
         "Returns a tuple with (start, end) of the wanted result"
         return self.index[index]
 
-    def uses_statistics(self):
+    def uses_statistics(self) -> bool:
         return False
 
-    def write_to_csv(self, destination=sys.stdout, delimiter=" "):
+    def write_to_csv(self, destination: str | IO[str] = sys.stdout, delimiter: str = " ") -> None:
         """
         writes the data to a file or to sys.stdout
         destination can be a file or a filename
         suitable for further processing
         """
-        # write sorted
-        the_destination=destination
-        if type(destination) in (str,):
-            the_destination=open(destination, "w")
+        if isinstance(destination, str):
+            with open(destination, "w") as f:
+                self._write_to_csv_dest(f, delimiter)
+        else:
+            self._write_to_csv_dest(destination, delimiter)
 
-        the_destination.write("# adc_result\n")
-        the_destination.write("# t y0 y1 ...\n")
+    def _write_to_csv_dest(self, dest: IO[str], delimiter: str) -> None:
+        dest.write("# adc_result\n")
+        dest.write("# t y0 y1 ...\n")
         self.lock.acquire()
         try:
             xdata=self.get_xdata()
             ch_no=self.get_number_of_channels()
             ydata=list(map(self.get_ydata, range(ch_no)))
-            #yerr=map(self.get_yerr, xrange(ch_no))
             for i in range(len(xdata)):
-                the_destination.write("%e"%xdata[i])
+                dest.write("%e"%xdata[i])
                 for j in range(ch_no):
-                    the_destination.write("%s%e"%(delimiter, ydata[j][i]))
-                the_destination.write("\n")
-            the_destination=None
-            xdata=ydata=None
+                    dest.write("%s%e"%(delimiter, ydata[j][i]))
+                dest.write("\n")
         finally:
             self.lock.release()
 
-    def write_to_simpson(self, destination=sys.stdout, delimiter=" "):
+    def write_to_simpson(self, destination: str | IO[str] = sys.stdout, delimiter: str = " ") -> None:
         """
         writes the data to a text file or sys.stdout in Simpson format,
         for further processing with the NMRnotebook software;
         destination can be a file or a filename
         """
-        # write sorted
-        the_destination=destination
-        if type(destination) in (str,):
-            the_destination=open(destination, "w")
+        if isinstance(destination, str):
+            with open(destination, "w") as f:
+                self._write_to_simpson_dest(f, delimiter)
+        else:
+            self._write_to_simpson_dest(destination, delimiter)
 
+    def _write_to_simpson_dest(self, dest: IO[str], delimiter: str) -> None:
         self.lock.acquire()
         try:
             xdata=self.get_xdata()
-            the_destination.write("SIMP\n")
-            the_destination.write("%s%i%s"%("NP=", len(xdata), "\n"))
-            the_destination.write("%s%i%s"%("SW=", self.get_sampling_rate(), "\n"))
-            the_destination.write("TYPE=FID\n")
-            the_destination.write("DATA\n")
+            dest.write("SIMP\n")
+            dest.write("%s%i%s"%("NP=", len(xdata), "\n"))
+            dest.write("%s%i%s"%("SW=", self.get_sampling_rate(), "\n"))
+            dest.write("TYPE=FID\n")
+            dest.write("DATA\n")
             ch_no=self.get_number_of_channels()
             ydata=list(map(self.get_ydata, range(ch_no)))
             for i in range(len(xdata)):
                 for j in range(ch_no):
-                    the_destination.write("%g%s"%(ydata[j][i], delimiter))
-                the_destination.write("\n")
-            the_destination.write("END\n")
-            the_destination=None
-            xdata=ydata=None
+                    dest.write("%g%s"%(ydata[j][i], delimiter))
+                dest.write("\n")
+            dest.write("END\n")
         finally:
             self.lock.release()
 
-    def write_to_hdf(self, hdffile, where, name, title, complib=None, complevel=None):
+    def write_to_hdf(self, hdffile: tables.File, where: str, name: str, title: str, complib: str | None = None, complevel: int | None = None) -> None:
         accu_group=hdffile.create_group(where=where,name=name,title=title)
         accu_group._v_attrs.damaris_type="ADC_Result"
         if self.contains_data():
@@ -371,13 +471,13 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
 
     # Ueberladen von Operatoren und Built-Ins -------------------------------------------------------
 
-    def __len__(self):
+    def __len__(self) -> int:
         "Redefining len(ADC_Result obj), returns the number of samples in one channel and 0 without data"
         if len(self.y)>0:
             return len(self.y[0])
         return 0
 
-    def __repr__(self):
+    def __repr__(self) -> str:
         """
         writes job meta data and data to string returned
         """
@@ -394,7 +494,12 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
 
         return tmp_string
 
-    def __add__(self, other):
+    def _copy_metadata(self, r: ADC_Result) -> None:
+        "Copy channel metadata (nChannels, is_clipped) from self to the new result r"
+        r.nChannels = self.nChannels
+        r.is_clipped = self.is_clipped
+
+    def __add__(self, other: int | float) -> ADC_Result:
         "Redefining self + other (scalar)"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -404,6 +509,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
                 tmp_y.append(numpy.array(self.y[i], dtype="float32") + other)
 
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
@@ -411,12 +517,12 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
 
 
 
-    def __radd__(self, other):
+    def __radd__(self, other: int | float) -> ADC_Result:
         "Redefining other (scalar) + self"
         return self.__add__(other)
 
 
-    def __sub__(self, other):
+    def __sub__(self, other: int | float) -> ADC_Result:
         "Redefining self - other (scalar)"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -426,6 +532,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
                 tmp_y.append(numpy.array(self.y[i], dtype="float32") - other)
 
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
@@ -433,7 +540,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
 
 
 
-    def __rsub__(self, other):
+    def __rsub__(self, other: int | float) -> ADC_Result:
         "Redefining other (scalar) - self"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -443,6 +550,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
                 tmp_y.append(other - numpy.array(self.y[i], dtype="float32"))
 
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
@@ -450,7 +558,7 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
 
 
 
-    def __mul__(self, other):
+    def __mul__(self, other: int | float) -> ADC_Result:
         "Redefining self * other (scalar)"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -460,18 +568,19 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
                 tmp_y.append(numpy.array(self.y[i], dtype="float32") * other)
 
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
             raise ValueError(f"ValueError: Cannot multiply \"{other.__class__}\" to ADC-Result!")
 
 
-    def __rmul__(self, other):
+    def __rmul__(self, other: int | float) -> ADC_Result:
         "Redefining other (scalar) * self"
         return self.__mul__(other)
 
 
-    def __pow__(self, other):
+    def __pow__(self, other: int | float) -> ADC_Result:
         "Redefining self ** other (scalar)"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -481,12 +590,13 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
                 tmp_y.append(numpy.array(self.y[i], dtype="float32") ** other)
 
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
             raise ValueError(f"ValueError: Cannot power raise \"{other.__class__}\" to ADC-Result!")
 
-    def __truediv__(self, other):
+    def __truediv__(self, other: int | float) -> ADC_Result:
         "Redefining self / other (scalar)"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -495,12 +605,13 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
                 tmp_y.append(numpy.array(self.y[i], dtype="float32") / other)
 
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to ADC-Result!")
 
-    def __rtruediv__(self, other):
+    def __rtruediv__(self, other: int | float) -> ADC_Result:
         "Redefining other (scalar) / self"
         if isinstance(other, int) or isinstance(other, float):
             self.lock.acquire()
@@ -508,45 +619,48 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
             for i in range(self.get_number_of_channels()):
                 tmp_y.append(other / numpy.array(self.y[i], dtype="float32"))
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to ADC-Result!")
 
-    def __floordiv__(self, other):
+    def __floordiv__(self, other: int) -> ADC_Result:
         "Redefining self // other (scalar)"
         if isinstance(other, float):
             raise ValueError("ValueError: Cannot use floor division (//) on floats! Use \"/\" instead of \"//\"! ")
 
         if isinstance(other, int):
             self.lock.acquire()
-            tmp_y = []
+            tmp_y: list[numpy.ndarray] = []
             for i in range(self.get_number_of_channels()):
                 tmp_y.append(numpy.array(self.y[i], dtype="float32") // other)
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to ADC-Result!")
 
-    def __rfloordiv__(self, other):
+    def __rfloordiv__(self, other: int) -> ADC_Result:
         "Redefining other (scalar) // self"
         if isinstance(other, float):
             raise ValueError("ValueError: Cannot use floor division (//) on floats! Use \"/\" instead of \"//\"! ")
 
         if isinstance(other, int):
             self.lock.acquire()
-            tmp_y = []
+            tmp_y: list[numpy.ndarray] = []
             for i in range(self.get_number_of_channels()):
                 tmp_y.append(other // numpy.array(self.y[i], dtype="float32"))
             r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+            self._copy_metadata(r)
             self.lock.release()
             return r
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to ADC-Result!")
 
 
-    def __neg__(self):
+    def __neg__(self) -> ADC_Result:
         "Redefining -self"
         self.lock.acquire()
         tmp_y = []
@@ -555,11 +669,12 @@ class ADC_Result(Resultable, Drawable, DamarisFFT, Signalpath):
             tmp_y.append(numpy.array(-self.y[i]))
 
         r = ADC_Result(x = self.x[:], y = tmp_y, index = self.index[:], sampl_freq = self.sampling_rate, desc = self.description, job_id = self.job_id, job_date = self.job_date)
+        self._copy_metadata(r)
         self.lock.release()
         return r
 
 
-def read_from_hdf(hdf_node):
+def read_from_hdf(hdf_node: object) -> ADC_Result | None:
     """
     read accumulation data from HDF node and return it.
     """
@@ -609,6 +724,7 @@ def read_from_hdf(hdf_node):
     adc.cont_data=True
     # now do the real data
     adc_data=hdf_node.adc_data.read()
+    adc.set_nChannels(adc_data.shape[1])
 
     adc.x=numpy.arange(adc_data.shape[0], dtype="float32")/adc.sampling_rate
 

src/damaris/data/Accumulation.py

@@ -25,6 +25,12 @@ import datetime # added by Oleg Petrov
 import ctypes   # added by Oleg Petrov
 import struct   # added by Oleg Petrov
 import os       # added by Oleg Petrov
+from typing import IO
+
+_XArray = numpy.ndarray | list
+_YChannels = list[numpy.ndarray]
+_IndexList = list[tuple[int, int]]
+_Description = dict[str, str]
 
 
 class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
@@ -50,7 +56,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
     x (list): The x-axis data points.
     y (list): The y-axis data points.
     """
-    def __init__(self, x = None, y = None, y_2 = None, n = None, index = None, sampl_freq = None, error = False):
+    def __init__(self, x: _XArray | None = None, y: _YChannels | None = None, y_2: _YChannels | numpy.ndarray | None = None, n: int | None = None, index: _IndexList | None = None, sampl_freq: float | None = None, error: bool = False) -> None:
         Errorable.__init__(self)
         Drawable.__init__(self)
         
@@ -64,14 +70,15 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         self.is_clipped = False
 
         self.common_descriptions=None
-        self.time_period=[]
+        self.time_period: list[datetime.datetime] | None = []
         self.job_id = None  # this does not make sense for an object accumulated from multiple job_ids
         self.job_ids = {}
 
         self.use_error = error
         if self.uses_statistics():
             if (y_2 is not None):
-                self.y_square = y_2
+                # y_square is always kept as a list of per-channel arrays
+                self.y_square: _YChannels = [numpy.asarray(ch) for ch in y_2]
             elif (y_2 is None) :
                 self.y_square = []
             else:
@@ -86,12 +93,12 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             
             self.cont_data = False
             self.index = []
-            self.x = []
+            self.x = numpy.array([], dtype="float32")
             self.y = []
 
         elif (x is not None) and (y is not None) and (index is not None) and (sampl_freq is not None) and (n is not None):
-            self.x = x
-            self.y = y
+            self.x = numpy.asarray(x)
+            self.y = [numpy.asarray(ch) for ch in y]
             self.sampling_rate = sampl_freq
 
             self.n = n
@@ -102,7 +109,15 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         else:
             raise ValueError("Wrong usage of __init__!")
 
-    def get_accu_by_index(self, index: int):
+    def _copy_provenance(self, r: "Accumulation") -> None:
+        r.is_clipped = self.is_clipped
+        r.job_id = self.job_id
+        r.job_ids = self.job_ids.copy()
+        r.time_period = self.time_period[:] if self.time_period is not None else None
+        if self.common_descriptions is not None:
+            r.common_descriptions = self.common_descriptions.copy()
+
+    def get_accu_by_index(self, index: int) -> "Accumulation":
         self.lock.acquire()
         try:
             start = self.index[index][0]
@@ -111,31 +126,32 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             self.lock.release()
             raise
 
-        tmp_x = self.x[start:end+1]
+        tmp_x = self.x[start:end+1].copy()
         tmp_y = []
 
         for i in range(self.get_number_of_channels()):
-            tmp_y.append(self.y[i][start:end+1])
+            tmp_y.append(self.y[i][start:end+1].copy())
 
-        r = Accumulation(x = tmp_x, y = tmp_y, n = self.n, index = [(0,len(tmp_y[0])-1)], sampl_freq = self.sampling_rate, error = self.use_error)            
+        r = Accumulation(x = tmp_x, y = tmp_y, n = self.n, index = [(0,len(tmp_y[0])-1)], sampl_freq = self.sampling_rate, error = self.use_error)
         if self.uses_statistics():
             r.y_square = []
             for i in range(self.get_number_of_channels()):
-                r.y_square.append(self.y_square[i][start:end+1])
+                r.y_square.append(self.y_square[i][start:end+1].copy())
         if self.common_descriptions is not None:
             r.common_descriptions = self.common_descriptions.copy()
-        r.time_period = self.time_period[:]
+        r.time_period = self.time_period[:] if self.time_period is not None else None
         r.job_ids = self.job_ids.copy()
+        r.is_clipped = self.is_clipped
         self.lock.release()
         return r
 
-    def get_result_by_index(self, index):
+    def get_result_by_index(self, index: int) -> "Accumulation":
         return self.get_accu_by_index(index)
 
-    def get_Result_by_index(self, index):
+    def get_Result_by_index(self, index: int) -> "Accumulation":
         return self.get_accu_by_index(index)
 
-    def get_ysquare(self, channel):
+    def get_ysquare(self, channel: int) -> numpy.ndarray | None:
         if self.uses_statistics():
             try:
                 return self.y_square[channel]
@@ -163,15 +179,15 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
 
     # external interface  --------------------------------------------------------------------
 
-    def get_yerr(self, channel):
+    def get_yerr(self, channel: int) -> numpy.ndarray:
         """
         return error (std.dev/sqrt(n)) of mean
         """
 
+        if not self.contains_data():
+            return numpy.array([], dtype="float32")
         if not self.uses_statistics():
             return numpy.zeros((len(self.y[0]),),dtype="float32")
-        if not self.contains_data():
-            return []
 
         self.lock.acquire()
         if self.n < 2:
@@ -187,13 +203,13 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         # sample standard deviation / sqrt(n)
         return numpy.nan_to_num(numpy.sqrt(variance_over_n))
 
-    def get_ydata(self, channel):
+    def get_ydata(self, channel: int = 0) -> numpy.ndarray:
         """
         return mean data
         """
 
         if not self.contains_data():
-            return []
+            return numpy.array([], dtype="float32")
         self.lock.acquire()
         try:
             tmp_y = self.y[channel] / self.n
@@ -205,7 +221,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         return tmp_y
 
 
-    def get_ymin(self):
+    def get_ymin(self) -> float|int:
         if not self.contains_data():
             return 0
         
@@ -240,86 +256,91 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         self.lock.release()
         return max(tmp_max)
 
-    def get_job_id(self) -> int:
+    def get_job_id(self) -> int | None:
         return self.job_id # modified by Oleg Petrov
 
-    def write_to_csv(self, destination=sys.stdout, delimiter=" "):
+    def write_to_csv(self, destination: str | IO[str] = sys.stdout, delimiter: str = " ") -> None:
         """
         writes the data to a file.
         destination can be a filehandle or a filename, default sys.stdout
         """
-        the_destination=destination
         if isinstance(destination, str):
-            the_destination=open(destination, "w")
+            with open(destination, "w") as f:
+                self._write_to_csv_dest(f, delimiter)
+        else:
+            self._write_to_csv_dest(destination, delimiter)
 
-        the_destination.write("# accumulation %d\n"%self.n)
+    def _write_to_csv_dest(self, dest: IO[str], delimiter: str) -> None:
+        dest.write("# accumulation %d\n"%self.n)
         self.lock.acquire()
         try:
             if self.common_descriptions is not None:
                 for (key,value) in self.common_descriptions.items():
-                    the_destination.write("# %s : %s\n"%(key, str(value)))
-            the_destination.write("# t")
+                    dest.write("# %s : %s\n"%(key, str(value)))
+            dest.write("# t")
             ch_no=self.get_number_of_channels()
             if self.use_error:
                 for i in range(ch_no):
-                    the_destination.write(" ch%d_mean ch%d_err"%(i,i))
+                    dest.write(" ch%d_mean ch%d_err"%(i,i))
             else:
                 for i in range(ch_no):
-                    the_destination.write(" ch%d_mean"%i)
-            the_destination.write("\n")
+                    dest.write(" ch%d_mean"%i)
+            dest.write("\n")
             xdata=self.get_xdata()
             ydata=list(map(self.get_ydata, range(ch_no)))
             yerr=None
             if self.use_error:
                 yerr=list(map(self.get_yerr, range(ch_no)))
             for i in range(len(xdata)):
-                the_destination.write("%e"%xdata[i])
+                dest.write("%e"%xdata[i])
                 for j in range(ch_no):
                     if self.use_error:
-                        the_destination.write("%s%e%s%e"%(delimiter, ydata[j][i], delimiter, yerr[j][i]))
+                        dest.write("%s%e%s%e"%(delimiter, ydata[j][i], delimiter, yerr[j][i]))
                     else:
-                        the_destination.write("%s%e"%(delimiter,ydata[j][i]))                        
-                the_destination.write("\n")
+                        dest.write("%s%e"%(delimiter,ydata[j][i]))
+                dest.write("\n")
         finally:
             self.lock.release()
             
 # ------------- added by Oleg Petrov, 14 Feb 2012 ----------------------
-    def write_to_simpson(self, destination=sys.stdout, delimiter=" ", frequency=100e6):
+    def write_to_simpson(self, destination: str | IO[str] = sys.stdout, delimiter: str = " ", frequency: float = 100e6) -> None:
         """
         writes the data to a text file or sys.stdout in Simpson format,
         for further processing with the NMRnotebook software;
         destination can be a file or a filename
         """
-        the_destination=destination
         if isinstance(destination, str):
-            the_destination=open(destination, "w")
+            with open(destination, "w") as f:
+                self._write_to_simpson_dest(f, delimiter, frequency)
+        else:
+            self._write_to_simpson_dest(destination, delimiter, frequency)
 
+    def _write_to_simpson_dest(self, dest: IO[str], delimiter: str, frequency: float) -> None:
         self.lock.acquire()
         try:
             xdata=self.get_xdata()
-            the_destination.write("SIMP\n")
-            the_destination.write("%s%i%s"%("NP=", len(xdata), "\n"))
-            the_destination.write("%s%i%s"%("SW=", self.get_sampling_rate(), "\n"))
-            the_destination.write("%s%i%s"%("REF=", frequency, "\n"))
-            the_destination.write("TYPE=FID\n")
-            the_destination.write("DATA\n")
+            dest.write("SIMP\n")
+            dest.write("%s%i%s"%("NP=", len(xdata), "\n"))
+            dest.write("%s%i%s"%("SW=", self.get_sampling_rate(), "\n"))
+            dest.write("%s%i%s"%("REF=", frequency, "\n"))
+            dest.write("TYPE=FID\n")
+            dest.write("DATA\n")
             ch_no=self.get_number_of_channels()
             ydata=list(map(self.get_ydata, range(ch_no)))
             for i in range(len(xdata)):
                 for j in range(ch_no):
-                    the_destination.write("%g%s"%(ydata[j][i], delimiter))
-                the_destination.write("\n")
-            the_destination.write("END\n")
-            the_destination.close()
+                    dest.write("%g%s"%(ydata[j][i], delimiter))
+                dest.write("\n")
+            dest.write("END\n")
         finally:
             self.lock.release()        
 
 # ------------- added by Oleg Petrov, 10 Sep 2013 -----------------------
-    def write_to_tecmag(self, destination=sys.stdout, nrecords=1,\
-                                                      frequency=100.,\
-                                                      last_delay = 1.,\
-                                                      receiver_phase=0.,\
-                                                      nucleus='1H'):
+    def write_to_tecmag(self, destination = sys.stdout, nrecords: int = 1,\
+                                                       frequency: float = 100.,\
+                                                       last_delay: float = 1.,\
+                                                       receiver_phase: float = 0.,\
+                                                       nucleus: str = '1H') -> None:
         """
         writes the data to a binary file in TecMag format;
         destination can be a file object or a filename;
@@ -348,6 +369,8 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             base_freq = [frequency, frequency, 0., 0.] 
             offset_freq = [0., 0., 0., 0.]
             ob_freq = [sum(x) for x in zip(base_freq, offset_freq)]
+            if self.time_period is None or len(self.time_period)==0:
+                raise ValueError("write_to_tecmag: no time information available")
             date = self.time_period[0].strftime("%Y/%m/%d %H:%M:%S")
 
             # data handling:
@@ -442,7 +465,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
 
 # -----------------------------------------------------------------------
 
-    def write_to_hdf(self, hdffile, where, name, title, complib=None, complevel=None):
+    def write_to_hdf(self, hdffile: tables.File, where: str, name: str, title: str, complib: str | None = None, complevel: int | None = None) -> None:
         """
         Write the accumulation data to an HDF5 file.
 
@@ -603,7 +626,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         return tmp_string
 
 
-    def __add__(self, other):
+    def __add__(self, other: "int | float | ADC_Result | Accumulation | None") -> "Accumulation":
         """
         Redefining self + other
         We define this for different types of other:
@@ -631,12 +654,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                     r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, y_2 = tmp_ysquare, n = self.n, index = self.index, sampl_freq = self.sampling_rate, error = self.use_error)
                 else:
                     r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, n = self.n, index = self.index, sampl_freq = self.sampling_rate, error = self.use_error)
-                r.is_clipped = self.is_clipped
-                r.job_id = self.job_id
-                r.job_ids = self.job_ids.copy()
-                r.time_period = self.time_period[:] if self.time_period is not None else None
-                if self.common_descriptions is not None:
-                    r.common_descriptions = self.common_descriptions.copy()
+                self._copy_provenance(r)
                 self.lock.release()
                 return r
 
@@ -663,7 +681,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                     r = Accumulation(x = numpy.array(other.x, dtype="float32"), y = tmp_y, index = other.index, sampl_freq = other.sampling_rate, n = 1, error = False)
                 if hasattr(other, "is_clipped"):
                     r.is_clipped = other.is_clipped
-                r.time_period=[other.job_date,other.job_date]
+                r.time_period=[other.job_date,other.job_date] if other.job_date is not None else None
                 r.job_id = other.job_id
                 r.common_descriptions=other.description.copy()
                 r.job_ids[other.job_id] = other.get_job_date()
@@ -672,8 +690,6 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
 
             # Other and self not empty (self + other ADC_Result)
             else:
-                self.lock.acquire()
-                
                 if self.sampling_rate != other.get_sampling_rate():
                     raise ValueError("Accumulation: You cant add ADC_Results with different sampling-rates")
                 if len(self.y[0]) != len(other):
@@ -684,6 +700,8 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                     if self.index[i] != other.get_index_bounds(i):
                         raise ValueError("Accumulation: You cant add ADC_Results with different indexing")
 
+                self.lock.acquire()
+                
                 tmp_y = []
                 tmp_ysquare = []
 
@@ -697,10 +715,13 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 else:
                     r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, n = self.n + 1, index = self.index, sampl_freq = self.sampling_rate, error = False)
                 r.is_clipped = self.is_clipped or (hasattr(other, "is_clipped") and other.is_clipped)
-                r.time_period=[min(self.time_period[0],other.job_date),
-                               max(self.time_period[1],other.job_date)]
+                if self.time_period and other.job_date is not None:
+                    r.time_period=[min(self.time_period[0],other.job_date),
+                                   max(self.time_period[1],other.job_date)]
+                else:
+                    r.time_period=None
                 r.job_id = other.job_id
-                r.job_ids = self.job_ids
+                r.job_ids = self.job_ids.copy()
                 r.job_ids[other.job_id] = other.get_job_date()
 
                 if self.common_descriptions is not None:
@@ -719,33 +740,29 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
 
             # Self empty (copy)
             if not self.contains_data():
+                if self.uses_statistics() and not other.uses_statistics():
+                    raise ValueError("Accumulation: You cant add non-error accumulations to accumulations with error")
+
+                self.lock.acquire()
 
                 tmp_y = []
                 tmp_ysquare = []
 
-                self.lock.acquire()
+                for i in range(other.get_number_of_channels()):
+                    tmp_y.append(numpy.array(other.y[i], dtype="float32"))
+                    if other.uses_statistics():
+                        tmp_ysquare.append(numpy.array(other.y_square[i], dtype="float32"))
 
                 if self.uses_statistics():
                     r = Accumulation(x = numpy.array(other.x, dtype="float32"), y = tmp_y, y_2 = tmp_ysquare, n = other.n, index = other.index, sampl_freq = other.sampling_rate, error = True)
                 else:
                     r = Accumulation(x = numpy.array(other.x, dtype="float32"), y = tmp_y, n = other.n, index = other.index, sampl_freq = other.sampling_rate, error = False)
-                for i in range(other.get_number_of_channels()):
-                    tmp_y.append(other.y[i])
-                    tmp_ysquare.append(other.y_square[i])
-                r.time_period=other.time_period[:]
-                r.job_id = other.job_id
-                r.job_ids = other.job_ids
-                if other.common_descriptions is not None:
-                    r.common_descriptions=other.common_descriptions.copy()
-                else:
-                    r.common_descriptions=None
+                other._copy_provenance(r)
                 self.lock.release()
                 return r
 
             # Other and self not empty (self + other)
             else:
-                self.lock.acquire()
-
                 if self.sampling_rate != other.get_sampling_rate():
                     raise ValueError("Accumulation: You cant add accumulations with different sampling-rates")
                 if len(self.y[0]) != len(other):
@@ -758,6 +775,8 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 if self.uses_statistics() and not other.uses_statistics():
                     raise ValueError("Accumulation: You cant add non-error accumulations to accumulations with error")
 
+                self.lock.acquire()
+
                 tmp_y = []
                 tmp_ysquare = []
 
@@ -771,9 +790,13 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 else:
                     r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, n = other.n + self.n, index = self.index, sampl_freq = self.sampling_rate, error = False)
                 r.is_clipped = self.is_clipped or (hasattr(other, "is_clipped") and other.is_clipped)
-                r.time_period=[min(self.time_period[0],other.time_period[0]),
-                               max(self.time_period[1],other.time_period[1])]
+                if self.time_period and other.time_period:
+                    r.time_period=[min(self.time_period[0],other.time_period[0]),
+                                   max(self.time_period[1],other.time_period[1])]
+                else:
+                    r.time_period=None
                 r.job_id = other.job_id
+                r.job_ids = self.job_ids.copy()
                 for i in other.job_ids:
                     r.job_ids[i] = other.job_ids[i]
                 r.common_descriptions={}
@@ -787,22 +810,28 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 return r
             
 
-    def __radd__(self, other):
+        elif other is None:
+            # Convenience: ignore add of None
+            return self + 0
+        else:
+            raise ValueError("can not add "+repr(type(other))+" to Accumulation")
+
+    def __radd__(self, other: "int | float | ADC_Result | Accumulation | None") -> "Accumulation":
         """Redefining other + self"""
         return self.__add__(other)
 
 
-    def __sub__(self, other):
+    def __sub__(self, other: "int | float | ADC_Result | Accumulation") -> "Accumulation":
         """Redefining self - other"""
         return self.__add__(-other)
             
 
-    def __rsub__(self, other):
+    def __rsub__(self, other: "int | float | ADC_Result | Accumulation") -> "Accumulation":
         """Redefining other - self"""
         return (-self) + other
 
 
-    def __mul__(self, other):
+    def __mul__(self, other: int | float) -> "Accumulation":
         """Redefining self * other (scalar)"""
         if isinstance(other, (int, float)):
             if not self.contains_data():
@@ -829,11 +858,11 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         else:
             raise ValueError(f"ValueError: Cannot multiply \"{other.__class__}\" to Accumulation!")
 
-    def __rmul__(self, other):
+    def __rmul__(self, other: int | float) -> "Accumulation":
         """Redefining other (scalar) * self"""
         return self.__mul__(other)
 
-    def __imul__(self, other):
+    def __imul__(self, other: int | float) -> "Accumulation":
         """Redefining self *= other (scalar)"""
         if isinstance(other, (int, float)):
             if not self.contains_data():
@@ -848,7 +877,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         else:
             raise ValueError(f"ValueError: Cannot multiply \"{other.__class__}\" to Accumulation!")
 
-    def __truediv__(self, other):
+    def __truediv__(self, other: int | float) -> "Accumulation":
         """Redefining self / other (scalar)"""
         if isinstance(other, (int, float)):
             if not self.contains_data():
@@ -875,7 +904,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to Accumulation!")
 
-    def __itruediv__(self, other):
+    def __itruediv__(self, other: int | float) -> "Accumulation":
         """Redefining self /= other (scalar)"""
         if isinstance(other, (int, float)):
             if not self.contains_data():
@@ -890,7 +919,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to Accumulation!")
 
-    def __floordiv__(self, other):
+    def __floordiv__(self, other: int) -> "Accumulation":
         """Redefining self // other (scalar)"""
         if isinstance(other, float):
             raise ValueError("ValueError: Cannot use floor division (//) on floats! Use \"/\" instead of \"//\"! ")
@@ -899,7 +928,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             if not self.contains_data():
                 raise ValueError("Accumulation: You cant divide an empty accumulation")
             self.lock.acquire()
-            tmp_y = []
+            tmp_y: list[numpy.ndarray] = []
             for i in range(self.get_number_of_channels()):
                 tmp_y.append(self.y[i] // other)
 
@@ -915,7 +944,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
         else:
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to Accumulation!")
 
-    def __ifloordiv__(self, other):
+    def __ifloordiv__(self, other: int) -> "Accumulation":
         """Redefining self //= other (scalar)"""
         if isinstance(other, float):
             raise ValueError("ValueError: Cannot use floor division (//) on floats! Use \"/\" instead of \"//\"! ")
@@ -936,7 +965,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             raise ValueError(f"ValueError: Cannot divide \"{other.__class__}\" to Accumulation!")
 
 
-    def __iadd__(self, other):
+    def __iadd__(self, other: "int | float | ADC_Result | Accumulation | None") -> "Accumulation":
         """Redefining self += other"""
         # Float or int
         if isinstance(other, int) or isinstance(other, float):
@@ -976,7 +1005,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
 
                 if hasattr(other, "is_clipped"):
                     self.is_clipped = other.is_clipped
-                self.time_period=[other.job_date,other.job_date]
+                self.time_period=[other.job_date,other.job_date] if other.job_date is not None else None
                 self.job_id = other.job_id      # added by Oleg Petrov
                 self.job_ids[other.job_id] = other.get_job_date()
                 self.common_descriptions=other.description.copy()
@@ -986,8 +1015,6 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
 
             # Other and self not empty (self + other/ADC_Result)
             else:
-                self.lock.acquire()
-
                 if self.sampling_rate != other.get_sampling_rate():
                     raise ValueError("Accumulation: You can't add ADC-Results with different sampling-rates")
                 if len(self.y[0]) != len(other):
@@ -998,6 +1025,8 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                     if self.index[i] != other.get_index_bounds(i):
                         raise ValueError("Accumulation: You can't add ADC-Results with different indexing")
 
+                self.lock.acquire()
+
                 for i in range(self.get_number_of_channels()):
                     self.y[i] += other.y[i]
                     if self.uses_statistics():
@@ -1006,8 +1035,11 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 self.n += 1
                 if hasattr(other, "is_clipped"):
                     self.is_clipped = self.is_clipped or other.is_clipped
-                self.time_period=[min(self.time_period[0],other.job_date),
-                                  max(self.time_period[1],other.job_date)]
+                if self.time_period and other.job_date is not None:
+                    self.time_period=[min(self.time_period[0],other.job_date),
+                                      max(self.time_period[1],other.job_date)]
+                else:
+                    self.time_period=None
 
                 self.job_id = other.job_id
 
@@ -1047,7 +1079,7 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 if hasattr(other, "is_clipped"):
                     self.is_clipped = other.is_clipped
                 self.common_descriptions=other.common_descriptions.copy()
-                self.time_period=other.time_period[:]
+                self.time_period=other.time_period[:] if other.time_period is not None else None
                 self.job_id = other.job_id      # added by Oleg Petrov
                 self.job_ids.update(other.job_ids)
                 self.lock.release()
@@ -1056,7 +1088,6 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             
             # Other and self not empty (self + other)
             else:
-                self.lock.acquire()
                 if self.sampling_rate != other.get_sampling_rate():
                     raise ValueError("Accumulation: You cant add accumulations with different sampling-rates")
                 if len(self.y[0]) != len(other):
@@ -1068,6 +1099,8 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                         raise ValueError("Accumulation: You cant add accumulations with different indexing")
                 if self.uses_statistics() and not other.uses_statistics():
                     raise ValueError("Accumulation: You cant add non-error accumulations to accumulations with error")
+
+                self.lock.acquire()
                     
                 for i in range(self.get_number_of_channels()):
                     self.y[i] += other.y[i]
@@ -1077,8 +1110,11 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
                 self.n += other.n
                 if hasattr(other, "is_clipped"):
                     self.is_clipped = self.is_clipped or other.is_clipped
-                self.time_period=[min(self.time_period[0],other.time_period[0]),
-                                  max(self.time_period[1],other.time_period[1])]
+                if self.time_period and other.time_period:
+                    self.time_period=[min(self.time_period[0],other.time_period[0]),
+                                      max(self.time_period[1],other.time_period[1])]
+                else:
+                    self.time_period=None
                 self.job_id = other.job_id
                 self.job_ids.update(other.job_ids)
                 # Removes mismatched common description keys
@@ -1108,16 +1144,16 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             raise ValueError("can not add "+repr(type(other))+" to Accumulation")
 
 
-    def __isub__(self, other):
+    def __isub__(self, other: "int | float | ADC_Result | Accumulation") -> "Accumulation":
         """Redefining self -= other"""
         return self.__iadd__(-other)
         
 
-    def __neg__(self):
+    def __neg__(self) -> "Accumulation":
         """Redefining -self"""
 
         if not self.contains_data():
-            return
+            raise ValueError("Accumulation: You cant negate an empty accumulation")
 
         tmp_y = []
 
@@ -1126,14 +1162,16 @@ class Accumulation(Errorable, Drawable, DamarisFFT, Signalpath):
             tmp_y.append(numpy.array(-self.y[i], dtype="float32"))
 
         if self.uses_statistics():
-            r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, y_2 = numpy.array(self.y_square), n = self.n, index = self.index, sampl_freq = self.sampling_rate, error = True)
+            tmp_y_square = [numpy.array(sq) for sq in self.y_square]
+            r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, y_2 = tmp_y_square, n = self.n, index = self.index, sampl_freq = self.sampling_rate, error = True)
         else:
             r = Accumulation(x = numpy.array(self.x, dtype="float32"), y = tmp_y, n = self.n, index = self.index, sampl_freq = self.sampling_rate, error = False)
+        self._copy_provenance(r)
         self.lock.release()
         return r
 
 
-def read_from_hdf(hdf_node):
+def read_from_hdf(hdf_node: object) -> "Accumulation | None":
     """
     read accumulation data from HDF node and return it.
     """

src/damaris/data/Config_Result.py

@@ -17,8 +17,12 @@ class Config_Result(Resultable):
 
         if config is None:
             self.config = { }
+        else:
+            self.config = config
         if desc is None:
             self.description = { }
+        else:
+            self.description = desc
         self.job_id = job_id
         self.job_date = job_date
 

src/damaris/data/DamarisFFT.py

@@ -19,10 +19,6 @@ class DamarisFFT:
         :param float stop: end of clipping in s
 
         """
-        # check if start/stop order is properly
-        if start > stop:
-            start, stop = stop, start
-
         # do nothing if one uses clip as a "placeholder"
         if start is None and stop is None:
             return self
@@ -32,6 +28,10 @@ class DamarisFFT:
         if stop is None:
             stop = self.x[ -1 ]
 
+        # check if start/stop order is properly
+        if start > stop:
+            start, stop = stop, start
+
         # check if data is fft which changes the start/stop units
         if self.xlabel == "Frequency / Hz":
             start = self.x.size * (0.5 + start / self.sampling_rate)

src/damaris/data/DataPool.py

@@ -12,11 +12,11 @@ import numpy
 class DataPool(collections.abc.MutableMapping):
     """
     dictionary with sending change events
-    """
-
-    # supports tranlation from dictionary keys to pytables hdf node names
+    
+    # supports translation from dictionary keys to pytables hdf node names
     # taken from: Python Ref Manual Section 2.3: Identifiers and keywords
     # things are always prefixed by "dir_" or "dict_"
+    """
     translation_table=""
     for i in range(256):
         c=chr(i)
@@ -47,20 +47,20 @@ class DataPool(collections.abc.MutableMapping):
 
     def __init__(self):
         self.__mydict={}
-        self.__dictlock=threading.Lock()
-        self.__registered_listeners=[]
+        self._dictlock=threading.RLock()
+        self._registered_listeners=[]
         self.__dirty_keys=set()
 
     def __getitem__(self, name):
         try:
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             return self.__mydict[name]
         finally:
-            self.__dictlock.release()
+            self._dictlock.release()
 
     def __setitem__(self, name, value):
         try:
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             if name in self.__mydict:
                 e=DataPool.Event(DataPool.Event.updated_value,name,self)
             else:
@@ -68,49 +68,49 @@ class DataPool(collections.abc.MutableMapping):
             self.__mydict[name]=value
             self.__dirty_keys.add(name)
         finally:
-            self.__dictlock.release()
+            self._dictlock.release()
         self.__send_event(e)
 
 
     def __delitem__(self, name):
         try:
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             del self.__mydict[name]
             self.__dirty_keys.add(name)
         finally:
-            self.__dictlock.release()
+            self._dictlock.release()
         self.__send_event(DataPool.Event(DataPool.Event.deleted_key,name,self))
 
     def __iter__(self):
         try:
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             return iter(self.__mydict)
         finally:
-            self.__dictlock.release()
+            self._dictlock.release()
 
     def __len__(self):
         try:
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             return len(self.__mydict)
         finally:
-            self.__dictlock.release()
+            self._dictlock.release()
 
     def keys(self):
         try:
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             return list(self.__mydict.keys())
         finally:
-            self.__dictlock.release()
+            self._dictlock.release()
 
     def __send_event(self, _event):
-        for listeners in self.__registered_listeners:
+        for listeners in self._registered_listeners:
             listeners(_event.copy())
 
     def __del__(self):
         self.__send_event(DataPool.Event(DataPool.Event.destroy))
-        self.__registered_listeners=None
+        self._registered_listeners=None
 
-    def write_hdf5(self,hdffile,where="/",name="data_pool", complib=None, complevel=None, incremental=False):
+    def write_hdf5(self,hdffile,where="/",name="data_pool", complib=None, complevel=None, incremental=False, write_interval=0):
 
         """
         Writes the content of a DataPool object to an HDF5 file using the `tables` library.
@@ -143,6 +143,14 @@ class DataPool(collections.abc.MutableMapping):
             Compression level to use when applying compression. Has no effect if `complib`
             is set to None.
 
+        incremental : bool, optional
+            If True, only write keys that have been modified since the last write (dirty keys).
+            If False, write all keys. Defaults to False.
+
+        write_interval : int, optional
+            Time interval in seconds between periodic writes.
+            Defaults to 0.
+
         Raises:
         Exception
             If `hdffile` is neither a string nor a valid `tables.File` object.
@@ -170,13 +178,14 @@ class DataPool(collections.abc.MutableMapping):
         else:
             dump_group = dump_file.create_group(where, name, "DAMARIS data pool")
 
-        self.__dictlock.acquire()
+        self._dictlock.acquire()
         if incremental:
             dict_keys=list(self.__dirty_keys)
         else:
             # Full sync: current keys + any other dirty keys (e.g. deletions)
             dict_keys=list(set(self.__mydict.keys()) | self.__dirty_keys)
-        self.__dictlock.release()
+        
+        self._dictlock.release()
 
         try:
             for key in dict_keys:
@@ -204,34 +213,40 @@ class DataPool(collections.abc.MutableMapping):
                 group_keyname="dict_"+str(namelist[-1]).translate(DataPool.translation_table)
 
                 # find existing node for this key
+                node_found = False
                 if group_keyname in dump_dir:
                     if dump_dir._v_children[group_keyname]._v_title == key:
-                        dump_file.remove_node(dump_dir, group_keyname, recursive=True)
+                        node_found = True
                     else:
                         extension_count = 0
                         while group_keyname + "_%03d" % extension_count in dump_dir:
                             suffix_name = group_keyname + "_%03d" % extension_count
                             if dump_dir._v_children[suffix_name]._v_title == key:
-                                dump_file.remove_node(dump_dir, suffix_name, recursive=True)
                                 group_keyname = suffix_name
+                                node_found = True
                                 break
                             extension_count += 1
                         else:
                             # truly new key with collision
                             group_keyname += "_%03d" % extension_count
 
-                self.__dictlock.acquire()
+                if node_found:
+                    dump_file.remove_node(dump_dir, group_keyname, recursive=True)
+
+                self._dictlock.acquire()
                 if key not in self.__mydict:
                     # Key was deleted
                     if key in self.__dirty_keys:
                         self.__dirty_keys.remove(key)
-                    self.__dictlock.release()
+                    self._dictlock.release()
                     continue
                 value=self.__mydict[key]
-                if key in self.__dirty_keys:
-                    self.__dirty_keys.remove(key)
-                self.__dictlock.release()
+                # We will clear the dirty flag only after successful write
+                # to handle potential transient failures or re-dirtied items
+                self._dictlock.release()
+                
                 # now write data, assuming, the object is constant during write operation
+                write_success = False
                 if hasattr(value, "write_to_hdf"):
                     try:
                         value.write_to_hdf(hdffile=dump_file,
@@ -240,6 +255,7 @@ class DataPool(collections.abc.MutableMapping):
                                            title=key,
                                            complib=complib,
                                            complevel=complevel)
+                        write_success = True
                     except Exception as e:
                         print("failed to write data_pool[\"%s\"]: %s"%(key,str(e)))
                         traceback_file=io.StringIO()
@@ -255,12 +271,24 @@ class DataPool(collections.abc.MutableMapping):
                                               name=group_keyname,
                                               obj=obj_to_save,
                                               title=key)
+                        write_success = True
                     except Exception as e:
                         print("failed to write data_pool[\"%s\"]: %s"%(key,str(e)))
                 else:
                     print("don't know how to store data_pool[\"%s\"] (type: %s)"%(key, type(value)))
+
+                if write_success:
+                    self._dictlock.acquire()
+                    if key in self.__dirty_keys:
+                        # Only remove if it hasn't been modified again in the meantime
+                        # (Checking identity is a simple way for objects)
+                        if self.__mydict.get(key) is value:
+                            self.__dirty_keys.remove(key)
+                    self._dictlock.release()
                 value=None
 
+            dump_file.flush()
+
         finally:
             dump_group=None
             if isinstance(hdffile, (bytes, str)):
@@ -296,9 +324,9 @@ class DataPool(collections.abc.MutableMapping):
             self._read_hdf_group(data_pool_group, "")
             
             # Clear dirty keys as they are now in sync with HDF5
-            self.__dictlock.acquire()
+            self._dictlock.acquire()
             self.__dirty_keys.clear()
-            self.__dictlock.release()
+            self._dictlock.release()
             
         finally:
             if close_file:
@@ -366,6 +394,12 @@ class DataPool(collections.abc.MutableMapping):
                         if obj is not None:
                             self[full_path] = obj
 
+                    elif damaris_type == "FitResult":
+                        from damaris.data.FitResult import read_from_hdf
+                        obj = read_from_hdf(node)
+                        if obj is not None:
+                            self[full_path] = obj
+
                     elif damaris_type == "TemperatureResult":
                         from damaris.data.Temperature import read_from_hdf
                         obj = read_from_hdf(node)
@@ -507,8 +541,8 @@ class DataPool(collections.abc.MutableMapping):
                 hdf_file.close()
 
     def register_listener(self, listening_function):
-        self.__registered_listeners.append(listening_function)
+        self._registered_listeners.append(listening_function)
 
     def unregister_listener(self, listening_function):
-        if listening_function in self.__registered_listeners:
-            self.__registered_listeners.remove(listening_function)
+        if listening_function in self._registered_listeners:
+            self._registered_listeners.remove(listening_function)

src/damaris/data/Drawable.py

@@ -1,5 +1,7 @@
 # -*- coding: iso-8859-1 -*-
 
+from __future__ import annotations
+
 #############################################################################
 #                                                                           #
 # Name: Class Drawable                                                      #
@@ -8,23 +10,29 @@
 #                                                                           #
 #############################################################################
 
+import numpy
+
+
 class Drawable:
     def __init__(self):
 
         # Will be set correctly in one of the subclasses
-        self.x = []
-        self.y = []
+        # x is always a numpy.ndarray, y is always a list of numpy.ndarray
+        # (one array per channel); empty data is represented by zero-length
+        # arrays of the same types
+        self.x: numpy.ndarray = numpy.array([], dtype="float64")
+        self.y: list[numpy.ndarray] = []
 
-        self.styles = { }
+        self.styles: dict = { }
 
-        self.xlabel = None
-        self.ylabel = None
+        self.xlabel: str | None = None
+        self.ylabel: str | None = None
 
-        self.title = None
+        self.title: str | None = None
 
-        self.legend = { }
+        self.legend: dict = { }
 
-        self.text = {}
+        self.text: dict = {}
 
         self.xmin = 0
         self.xmax = 0
@@ -32,12 +40,12 @@ class Drawable:
         self.ymax = 0
 
 
-    def get_xdata(self):
+    def get_xdata(self) -> numpy.ndarray:
         "Returns a reference to the x-Plotdata (array)"
         return self.x
 
 
-    def set_xdata(self, pos, value):
+    def set_xdata(self, pos: int, value: float) -> None:
         "Sets a point in x"
         try:
             self.x[pos] = value
@@ -45,7 +53,7 @@ class Drawable:
             raise
 
 
-    def get_ydata(self, channel):
+    def get_ydata(self, channel: int = 0) -> numpy.ndarray:
         "Returns the y-Plotdata of channel n (array)"
         try:
             return self.y[channel]
@@ -53,51 +61,52 @@ class Drawable:
             raise
 
 
-    def set_ydata(self, channel, pos, value):
+    def set_ydata(self, channel: int, pos: int, value: float) -> None:
         "Sets a point in y"
         try:
             self.y[channel][pos] = value
         except:
             raise
 
-    def get_number_of_channels(self):
+
+    def get_number_of_channels(self) -> int:
         "Returns the number of channels in y"
         return len(self.y)
 
 
-    def get_style(self):
+    def get_style(self) -> dict:
         "Returns a reference to plot-styles (dictionary)"
         return self.styles
 
 
-    def set_style(self, channel, value):
+    def set_style(self, channel: int | str, value: object) -> None:
         "Sets a channel to a certain plot-style"
         if channel in self.styles:
             print("Drawable Warning: Style key \"%s\" will be overwritten with \"%s\"" % (str(channel), str(value)))
         self.styles[channel] = str(value)
 
 
-    def get_xlabel(self):
+    def get_xlabel(self) -> str | None:
         "Returns the label for the x-axis"
         return self.xlabel
 
 
-    def set_xlabel(self, label):
+    def set_xlabel(self, label: object) -> None:
         "Sets the label for the x-axis"
         self.xlabel = str(label)
 
 
-    def get_ylabel(self):
+    def get_ylabel(self) -> str | None:
         "Gets the label for the y-axis"
         return self.ylabel
 
 
-    def set_ylabel(self, label):
+    def set_ylabel(self, label: object) -> None:
         "Sets the label for the y-axis"
         self.ylabel = str(label)
 
 
-    def get_text(self, index):
+    def get_text(self, index: int | str) -> str | None:
         "Returns labels to be plotted (List)"
         if index in self.text:
             return self.text[index]
@@ -105,84 +114,96 @@ class Drawable:
             return None
 
 
-    def set_text(self, index, text):
+    def set_text(self, index: int | str, text: object) -> None:
         "Sets labels to be plotted "
         self.text[index] = str(text)
 
 
-    def get_title(self):
+    def get_title(self) -> str | None:
         "Returns the title of the plot"
         return self.title
 
 
-    def set_title(self, title):
+    def set_title(self, title: object) -> None:
         "Sets the title of the plot"
         self.title = str(title)
 
 
-    def get_legend(self):
+    def get_legend(self) -> dict:
         "Returns the legend of the plot (Dictionary)"
         return self.legend
 
 
-    def set_legend(self, channel, value):
+    def set_legend(self, channel: int | str, value: object) -> None:
         "Sets the legend of the plot"
         if channel in self.legend:
             print("Drawable Warning: Legend key \"%s\" will be overwritten with \"%s\"" % (str(channel), str(value)))
         self.legend[channel] = str(value)
 
-    def get_xmin(self):
+
+    def get_xmin(self) -> float:
         "Returns minimun of x"
+        if self.x.size == 0:
+            return 0
         return self.x.min()
 
-    def get_xminpos(self):
+
+    def get_xminpos(self) -> float:
         "Returns smallest positive value of x"
         mask = self.x > 0
+        if not numpy.any(mask):
+            return 0
         return self.x[mask].min()
 
-    def set_xmin(self, xmin):
+
+    def set_xmin(self, xmin: float) -> None:
         "Sets minimum of x"
         self.xmin = xmin
 
-    def get_xmax(self):
+
+    def get_xmax(self) -> float:
         "Returns maximum of x"
+        if self.x.size == 0:
+            return 0
         return self.x.max()
 
-    def set_xmax(self, xmax):
+
+    def set_xmax(self, xmax: float) -> None:
         "Sets maximum of x"
         self.xmax = xmax
 
-    def get_ymin(self):
+
+    def get_ymin(self) -> float | int:
         "Returns minimum of y"
-        if isinstance(self.y, list):
-            return min([yarr.min() for yarr in self.y])
-        else:
-            return self.y.min()
+        if len(self.y) == 0:
+            return 0
+        return min([yarr.min() for yarr in self.y])
 
-    def get_yminpos(self):
+
+    def get_yminpos(self) -> float | int:
         "Returns smallest positive value of y"
-        if isinstance(self.y, list):
-            ymins = []
-            for ys in self.y:
-                mask = ys > 0
+        ymins = []
+        for ys in self.y:
+            mask = ys > 0
+            if numpy.any(mask):
                 ymins.append(ys[mask].min())
-            ymin = min(ymins)
-        else:
-            mask = self.y > 0
-            ymin = self.y[mask].min()
-        return ymin
+        if len(ymins) == 0:
+            return 0
+        return min(ymins)
 
-    def set_ymin(self, ymin):
+
+    def set_ymin(self, ymin: float) -> None:
         "Sets minimum of y"
         self.ymin = ymin
 
-    def get_ymax(self):
-        "Returns maximimum of y"
-        if isinstance(self.y, list):
-            return max([yarr.max() for yarr in self.y])
-        else:
-            return self.y.max()
 
-    def set_ymax(self, ymax):
+    def get_ymax(self) -> float | int:
+        "Returns maximimum of y"
+        if len(self.y) == 0:
+            return 0
+        return max([yarr.max() for yarr in self.y])
+
+
+    def set_ymax(self, ymax: float) -> None:
         "Sets maximum of y"
         self.ymax = ymax

src/damaris/data/Error_Result.py

@@ -2,6 +2,7 @@
 
 from .Resultable import Resultable
 from .Drawable import Drawable
+import numpy
 
 #############################################################################
 #                                                                           #
@@ -46,12 +47,12 @@ class Error_Result(Resultable, Drawable):
         return False
 
     # Nothing to plot
-    def get_ydata(self):
-        return [0.0]
+    def get_ydata(self, channel=0):
+        return numpy.array([0.0])
 
     # Nothing to plot
     def get_xdata(self):
-        return [0.0]
+        return numpy.array([0.0])
 
     #overload of operators und built-ins -------------------------------------------------------
 

src/damaris/data/FitResult.py

@@ -0,0 +1,127 @@
+# fit curve with x-values independent of the measurement data points
+
+import numpy
+import tables
+from . import Drawable
+
+
+class FitResult(Drawable.Drawable):
+    """
+    plottable fit curve belonging to a MeasurementResult
+
+    In contrast to MeasurementResult, the x-values are a plain array
+    and are independent of the measured data points, so a fit can be
+    sampled on a much denser x-grid for a smooth plot. The fit is
+    stored in the data pool under its own key, e.g. as a child of the
+    measurement ("Magnetization(tau)/fit"), and is drawn by the GUI
+    as an overlay on the measurement it belongs to.
+    """
+
+    def __init__(self, x, y, title=None, source=None, y_err=None, params=None):
+        """
+        x: array-like of x-values (independent of the measurement)
+        y: array-like of y-values, same length as x
+        title: plot title of the fit
+        source: data pool key of the MeasurementResult this fit belongs to
+        y_err: optional array-like of y errors, same length as x
+        params: optional dictionary of fit parameters, shown in the plot
+        """
+        Drawable.Drawable.__init__(self)
+        self.x = numpy.array(x, dtype="float64").reshape(-1)
+        self.y = [numpy.array(y, dtype="float64").reshape(-1)]
+        if y_err is not None:
+            self.y_err = numpy.array(y_err, dtype="float64").reshape(-1)
+        else:
+            self.y_err = None
+        self.title = title
+        self.source = source
+        self.params = dict(params) if params is not None else {}
+
+    def get_lineplotdata(self):
+        """
+        x and y data for plotting the fit curve
+        """
+        return [self.x, self.y[0]]
+
+    def uses_statistics(self):
+        """
+        drawable interface method returns False
+        """
+        return False
+
+    def write_to_hdf(self, hdffile, where, name, title, complib=None, complevel=None):
+        h5_table_format = {
+            "x": tables.Float64Col(),
+            "y": tables.Float64Col()
+            }
+        if self.y_err is not None:
+            h5_table_format["y_err"] = tables.Float64Col()
+        filter = None
+        if complib is not None:
+            if complevel is None:
+                complevel = 9
+            filter = tables.Filters(complevel=complevel, complib=complib, shuffle=1)
+
+        fit_table = hdffile.create_table(where=where, name=name,
+                                         description=h5_table_format,
+                                         title=title,
+                                         filters=filter,
+                                         expectedrows=self.x.shape[0])
+        fit_table.flavor = "numpy"
+        fit_table.attrs.damaris_type = "FitResult"
+        fit_table.attrs.quantity_name = self.get_title()
+        if self.xlabel is not None:
+            fit_table.attrs.xlabel = self.xlabel
+        if self.ylabel is not None:
+            fit_table.attrs.ylabel = self.ylabel
+        if self.source is not None:
+            fit_table.attrs.source = self.source
+        for (key, value) in self.params.items():
+            if isinstance(value, (int, float)) and not isinstance(value, bool):
+                fit_table.attrs.__setattr__("param_" + str(key), float(value))
+            else:
+                fit_table.attrs.__setattr__("param_" + str(key), str(value))
+
+        row = fit_table.row
+        has_y_err = self.y_err is not None
+        for i in range(self.x.shape[0]):
+            row["x"] = self.x[i]
+            row["y"] = self.y[0][i]
+            if has_y_err:
+                row["y_err"] = self.y_err[i]
+            row.append()
+        fit_table.flush()
+
+
+def read_from_hdf(hdf_node):
+    """
+    reads a FitResult object from the hdf_node
+    or None if the node is not suitable
+    """
+    if not isinstance(hdf_node, tables.Table):
+        return None
+
+    if hdf_node._v_attrs.damaris_type != "FitResult":
+        return None
+
+    xdata = hdf_node.col("x")
+    ydata = hdf_node.col("y")
+    if "y_err" in hdf_node.colnames:
+        y_err = hdf_node.col("y_err")
+    else:
+        y_err = None
+
+    fr = FitResult(xdata, ydata,
+                   title=getattr(hdf_node._v_attrs, "quantity_name", None),
+                   source=getattr(hdf_node._v_attrs, "source", None),
+                   y_err=y_err)
+    fr.xlabel = getattr(hdf_node._v_attrs, "xlabel", None)
+    fr.ylabel = getattr(hdf_node._v_attrs, "ylabel", None)
+
+    params = {}
+    for attr_name in hdf_node._v_attrs._f_list("all"):
+        if attr_name.startswith("param_"):
+            params[attr_name[6:]] = getattr(hdf_node._v_attrs, attr_name)
+    fr.params = params
+
+    return fr

src/damaris/data/MeasurementResult.py

@@ -1,10 +1,10 @@
 import threading
 import math
-import types
 import sys
 import tables
 import numpy
 import collections
+from typing import IO
 from . import Drawable
 
 ## provide gaussian statistics for a series of measured data points
@@ -13,7 +13,7 @@ from . import Drawable
 # internally it keeps the sum, the sum of squares and the number of data points
 class AccumulatedValue:
 
-    def __init__(self, mean=None, mean_err=None, n=None):
+    def __init__(self, mean: float | None = None, mean_err: float | None = None, n: int | None = None):
         """
         one value with std. deviation
         can be initialized by:
@@ -43,9 +43,9 @@ class AccumulatedValue:
             self.y=float(mean)*self.n
             self.y2=float(mean_err)**2*n*(n-1.0)+float(mean)**2*n
 
-    def __add__(self,y):
+    def __add__(self, y: "AccumulatedValue | float") -> "AccumulatedValue":
         new_one=AccumulatedValue()
-        if (type(y) is types.InstanceType and isinstance(y, AccumulatedValue)):
+        if isinstance(y, AccumulatedValue):
             new_one.y=self.y+y.y
             new_one.y2=self.y2+y.y2
             new_one.n=self.n+y.n
@@ -55,8 +55,8 @@ class AccumulatedValue:
             new_one.n=self.n+1
         return new_one
 
-    def __iadd__(self,y):
-        if (type(y) is types.InstanceType and isinstance(y, AccumulatedValue)):
+    def __iadd__(self, y: "AccumulatedValue | float") -> "AccumulatedValue":
+        if isinstance(y, AccumulatedValue):
             self.y+=y.y
             self.y2+=y.y2
             self.n+=y.n
@@ -66,14 +66,14 @@ class AccumulatedValue:
             self.n+=1
         return self
 
-    def copy(self):
+    def copy(self) -> "AccumulatedValue":
         a=AccumulatedValue()
         a.y=self.y
         a.y2=self.y2
         a.n=self.n
         return a
 
-    def mean(self):
+    def mean(self) -> float | None:
         """
         returns the mean of all added/accumulated values
         """
@@ -82,7 +82,7 @@ class AccumulatedValue:
         else:
             return self.y/self.n
 
-    def sigma(self):
+    def sigma(self) -> float | None:
         """
         returns the standard deviation added/accumulated values
         """
@@ -98,7 +98,7 @@ class AccumulatedValue:
         else:
             return None
 
-    def mean_error(self):
+    def mean_error(self) -> float | None:
         """
         returns the mean's error (=std.dev/sqrt(n)) of all added/accumulated values
         """
@@ -114,7 +114,7 @@ class AccumulatedValue:
         else:
             return None
 
-    def __str__(self):
+    def __str__(self) -> str:
         if self.n==0:
             return "no value"
         elif self.n==1:
@@ -122,12 +122,12 @@ class AccumulatedValue:
         else:
             return "%g +/- %g (%d accumulations)"%(self.mean(),self.mean_error(),self.n)
 
-    def __repr__(self):
+    def __repr__(self) -> str:
         return str(self)
 
 class MeasurementResult(Drawable.Drawable, collections.UserDict):
 
-    def __init__(self, quantity_name):
+    def __init__(self, quantity_name: str) -> None:
         """
         convenient accumulation and interface to plot functions
 
@@ -139,7 +139,7 @@ class MeasurementResult(Drawable.Drawable, collections.UserDict):
         self.lock = threading.RLock()
 
     # get the selected item, if it does not exist, create an empty one
-    def __getitem__(self, key):
+    def __getitem__(self, key: float) -> "AccumulatedValue":
         if key not in self:
             a=AccumulatedValue()
             self.data[float(key)]=a
@@ -147,23 +147,26 @@ class MeasurementResult(Drawable.Drawable, collections.UserDict):
         else:
             return self.data[float(key)]
 
-    def __setitem__(self,key,value):
-        if not isinstance(value, AccumulatedValue):
-            value=AccumulatedValue(float(value))
+    def __setitem__(self,key: float,item: "AccumulatedValue | float")-> None:
+        if not isinstance(item, AccumulatedValue):
+            item=AccumulatedValue(float(item))
         return collections.UserDict.__setitem__(self,
                                              float(key),
-                                             value)
+                                             item)
 
-    def __add__(self, right_value):
+    def __add__(self, right_value: int) -> "MeasurementResult":
         if right_value==0:
-            return self.copy()
+            new_one=MeasurementResult(self.quantity_name)
+            for key in self.data:
+                new_one[key]=self.data[key].copy()
+            return new_one
         else:
             raise Exception("not implemented")
 
-    def get_title(self):
+    def get_title(self) -> str:
         return self.quantity_name
 
-    def get_xdata(self):
+    def get_xdata(self) -> numpy.ndarray:
         """
         sorted array of all dictionary entries without Accumulated Value objects with n==0
         """
@@ -172,62 +175,61 @@ class MeasurementResult(Drawable.Drawable, collections.UserDict):
         keys.sort()
         return keys
 
-    def get_ydata(self):
+    def get_ydata(self, channel: int = 0) -> numpy.ndarray:
+        """
+        y data (mean values); the channel argument is accepted for
+        compatibility with the Drawable interface, MeasurementResult
+        is single-channel
+        """
         return self.get_xydata()[1]
 
-    def get_xydata(self):
+    def get_xydata(self) -> list[numpy.ndarray]:
         k=self.get_xdata()
         v=numpy.array([self.data[key].mean() for key in k], dtype="float64")
         return [k,v]
 
-    def get_errorplotdata(self):
+    def get_errorplotdata(self) -> list[numpy.ndarray]:
         k=self.get_xdata()
         v=numpy.array([self.data[key].mean() for key in k], dtype="float64")
         e=numpy.array([self.data[key].mean_error() for key in k], dtype="float64")
         return [k,v,e]
 
-    def get_lineplotdata(self):
-        k=self.get_xdata()
-        v=numpy.array(self.y, dtype="float64") 
-        return [k, v]
+    def get_lineplotdata(self) -> list[numpy.ndarray]:
+        return self.get_xydata()
 
-    def uses_statistics(self):
+    def uses_statistics(self) -> bool:
         """
         drawable interface method returns True
         """
         return True
 
-    def write_to_csv(self,destination=sys.stdout, delimiter=" "):
+    def write_to_csv(self,destination: str | IO[str] = sys.stdout, delimiter: str = " ") -> None:
         """
         writes the data to a file or to sys.stdout
         destination can be a file or a filename
         suitable for further processing
         """
-        the_destination=destination
-        file_opened = False
-        if type(destination) in (str,):
-            the_destination=open(destination, "w")
-            file_opened = True
+        if isinstance(destination, str):
+            with open(destination, "w") as f:
+                self._write_to_csv_dest(f, delimiter)
+        else:
+            self._write_to_csv_dest(destination, delimiter)
 
-        the_destination.write("# quantity:"+str(self.quantity_name)+"\n")
-        the_destination.write("# x y ysigma n\n")
+    def _write_to_csv_dest(self, dest: IO[str], delimiter: str) -> None:
+        dest.write("# quantity:"+str(self.quantity_name)+"\n")
+        dest.write("# x y ysigma n\n")
         for x in self.get_xdata():
-            y=self.data[x]
-            if type(y) in [float, int, int]:
-                the_destination.write("%e%s%e%s0%s1\n"%(x, delimiter, y, delimiter, delimiter))
-            else:
-                the_destination.write("%e%s%e%s%e%s%d\n"%(x,
-                                                          delimiter,
-                                                          y.mean(),
-                                                          delimiter,
-                                                          y.mean_error(),
-                                                          delimiter,
-                                                          y.n))
-        if file_opened:
-            the_destination.close()
+            y = self.data[x]
+            dest.write("%e%s%e%s%e%s%d\n" % (x,
+                                             delimiter,
+                                             y.mean(),
+                                             delimiter,
+                                             y.mean_error(),
+                                             delimiter,
+                                             y.n))
 
 
-    def write_to_hdf(self, hdffile, where, name, title, complib=None, complevel=None):
+    def write_to_hdf(self, hdffile: tables.File, where: str, name: str, title: str, complib: str | None = None, complevel: int | None = None) -> None:
         h5_table_format= {
             "x" : tables.Float32Col(),
             "y" : tables.Float32Col(),
@@ -253,24 +255,19 @@ class MeasurementResult(Drawable.Drawable, collections.UserDict):
             row=mr_table.row
             xdata=self.get_xdata()
             if xdata.shape[0]!=0:
-                for x in self.get_xdata():
+                for x in xdata:
                     y=self.data[x]
                     row["x"]=x
-                    if type(y) in [float, int, int]:
-                        row["y"]=y
-                        row["y_err"]=0.0
-                        row["n"]=1
-                    else:
-                        row["y"]=y.mean()
-                        row["y_err"]=y.mean_error()
-                        row["n"]=y.n
+                    row["y"]=y.mean()
+                    row["y_err"]=y.mean_error()
+                    row["n"]=y.n
                     row.append()
 
         finally:
             mr_table.flush()
             self.lock.release()
 
-def read_from_hdf(hdf_node):
+def read_from_hdf(hdf_node: object) -> "MeasurementResult | None":
     """
     reads a MeasurementResult object from the hdf_node
     or None if the node is not suitable

src/damaris/data/Signalpath.py

@@ -1,3 +1,5 @@
+import copy
+
 import numpy as N
 
 class Signalpath:
@@ -5,33 +7,35 @@ class Signalpath:
         """
         rotate signal by **degrees** for phase cycling, etc.
 
+        The signalpath object itself is not modified, a copy containing the
+        rotated data is returned.
+
         :param degrees: rotate signal by this value
-        :return:
+        :return: copy of the signalpath object with rotated data
         """
         if self.get_number_of_channels() != 2:
             raise Exception("rotation defined only for 2 channels")
+        rotated = copy.copy(self)
         # simple case 0, 90, 180, 270 degree
         reduced_angle = divmod(degrees, 90)
         if abs(reduced_angle[1]) < 1e-6:
             reduced_angle = reduced_angle[0] % 4
             if reduced_angle == 0:
-                return
+                rotated.y = [a.copy() for a in self.y]
 
             elif reduced_angle == 1:
-                self.y[1] *= -1
-                self.y = [self.y[1], self.y[0]]
+                rotated.y = [-self.y[1], self.y[0].copy()]
 
             elif reduced_angle == 2:
-                self.y[0] *= -1
-                self.y[1] *= -1
+                rotated.y = [-self.y[0], -self.y[1]]
 
             elif reduced_angle == 3:
-                self.y[0] *= -1
-                self.y = [self.y[1], self.y[0]]
+                rotated.y = [self.y[1].copy(), -self.y[0]]
         else:
-            sin_angle = N.sin(degrees / 180.0 * N.pi)
-            cos_angle = N.cos(degrees / 180.0 * N.pi)
-            self.y = [cos_angle * self.y[0] - sin_angle * self.y[1],
-                      sin_angle * self.y[0] + cos_angle * self.y[1]]
+            rad_angle = N.deg2rad(degrees)
+            sin_angle = N.sin(rad_angle)
+            cos_angle = N.cos(rad_angle)
+            rotated.y = [cos_angle * self.y[0] - sin_angle * self.y[1],
+                         sin_angle * self.y[0] + cos_angle * self.y[1]]
 
-        return self
+        return rotated

src/damaris/data/__init__.py

@@ -1,10 +1,11 @@
 from damaris.data.ADC_Result import ADC_Result
 from damaris.data.Accumulation import Accumulation
 from damaris.data.MeasurementResult import MeasurementResult, AccumulatedValue
+from damaris.data.FitResult import FitResult
 from damaris.data.DataPool import DataPool
 from damaris.data.Error_Result import Error_Result
 from damaris.data.Config_Result import Config_Result
 from damaris.data.Temperature import TemperatureResult
 
-__all__=["ADC_Result", "Accumulation", "MeasurementResult", "AccumulatedValue", "DataPool", "Error_Result", "Config_Result", "TemperatureResult" ]
+__all__=["ADC_Result", "Accumulation", "MeasurementResult", "AccumulatedValue", "FitResult", "DataPool", "Error_Result", "Config_Result", "TemperatureResult" ]
 

src/damaris/experiments/DurationEstimator.py

@@ -0,0 +1,109 @@
+import io
+import sys
+import traceback
+from damaris.experiments import Experiment
+
+
+class DurationEstimator:
+    """
+    Consumes an experiment script's generator and estimates total duration
+    without writing any files or invoking the backend.
+
+    Sleeps and synchronizations in the script are tracked instead of
+    blocking, so their wall-clock time is included in the estimate.
+    """
+
+    def __init__(self):
+        pass
+
+    def estimate(self, script):
+        """
+        Run the experiment script in a sandbox and return a duration estimate.
+
+        :param str script: experiment script source code
+        :returns: dict with 'experiment_time', 'sleep_time', 'total_time', and 'job_count'
+        :rtype: dict
+        """
+        sleep_time = [0.0]  # list so it's mutable in nested function
+
+        def tracking_sleep(seconds):
+            sleep_time[0] += seconds
+
+        def noop_sync(before=0, waitsteps=0.1):
+            pass
+
+        def noop_data():
+            return {}
+
+        dataspace = {}
+        exp_classes = __import__('damaris.experiments', dataspace, dataspace, ['Experiment'])
+        for name in dir(exp_classes):
+            if name[:2] == "__" and name[-2:] == "__":
+                continue
+            dataspace[name] = exp_classes.__dict__[name]
+        del exp_classes
+
+        # ranges module for lin_range, log_range, etc.
+        ranges = __import__('damaris.tools.ranges', dataspace, dataspace, [])
+        for name in dir(ranges):
+            if name[:2] == "__" and name[-2:] == "__":
+                continue
+            dataspace[name] = getattr(ranges, name)
+        del ranges
+
+        # inject sandbox helpers (non-blocking versions)
+        dataspace["data"] = noop_data
+        dataspace["synchronize"] = noop_sync
+        dataspace["sleep"] = tracking_sleep
+
+        # execute the script
+        try:
+            exec(script, dataspace)
+        except Exception as e:
+            traceback.print_exc()
+            raise RuntimeError("Script execution failed") from e
+
+        if "experiment" not in dataspace:
+            raise ValueError("Script does not define an experiment() function")
+
+        # consume the generator
+        try:
+            exp_iterator = dataspace["experiment"]()
+        except Exception as e:
+            traceback.print_exc()
+            raise RuntimeError("Calling experiment() failed") from e
+
+        experiment_time = 0.0
+        job_count = 0
+
+        while True:
+            try:
+                job = next(exp_iterator)
+            except StopIteration:
+                break
+            except Exception as e:
+                traceback.print_exc()
+                raise RuntimeError("Generator failed") from e
+
+            experiment_time += job.get_length()
+            job_count += 1
+
+        total_time = experiment_time + sleep_time[0]
+
+        return {
+            "experiment_time": experiment_time,
+            "sleep_time": sleep_time[0],
+            "total_time": total_time,
+            "job_count": job_count,
+        }
+
+
+def estimate_duration(script):
+    """
+    Convenience function to estimate experiment duration.
+
+    :param str script: experiment script source code
+    :returns: dict with 'experiment_time', 'sleep_time', 'total_time', and 'job_count'
+    :rtype: dict
+    """
+    return DurationEstimator().estimate(script)

src/damaris/experiments/Experiment.py

@@ -3,20 +3,20 @@ import types
 import numpy
 
 class StateBase(object):
-    def __init__(self):
+    def __init__(self) -> None:
         pass
-    def to_xml(self, indent = ""):
+    def to_xml(self, indent: str = "") -> str:
         return indent + "<!-- " + repr(self) + " -->"
 
 class StateSimple(StateBase):
-    def __init__(self, time, content=None):
+    def __init__(self, time: float, content: object | None = None) -> None:
         super(StateSimple, self).__init__()
         if time < 0:
             raise AssertionError("time for state is negative!")
-        self.time = time
+        self.time = float(time)
         self.content = content
 
-    def to_xml(self, indent = ""):
+    def to_xml(self, indent: str = "") -> str:
         s = indent + '<state time="%s"' % repr(self.time)
         if self.content is None:
             return s + '/>\n'
@@ -24,14 +24,14 @@ class StateSimple(StateBase):
         s += indent + '  ' + str(self.content) + '\n'
         s += indent + '</state>\n'
         return s
-    def __repr__(self):
+    def __repr__(self) -> str:
         return 'StateSimple(%s, %s)' % (self.time, repr(self.content))
 
 class StateList(StateBase):
-    def __init__(self):
+    def __init__(self) -> None:
         super(StateList, self).__init__()
-        self.list = []
-    def to_xml(self, indent = "  "):
+        self.list: list[object] = []
+    def to_xml(self, indent: str = "  ") -> str:
         s = ""
         for k in self.list:
             if hasattr(k, "to_xml"):
@@ -39,21 +39,21 @@ class StateList(StateBase):
             else:
                 s += indent + str(k)
         return s
-    def append(self, val):
+    def append(self, val: object) -> None:
         self.list.append(val)
 
 
 class StateLoop(StateList):
     """Represents a loop in the state tree"""
-    def __init__(self, repeat):
+    def __init__(self, repeat: int) -> None:
         super(StateLoop, self).__init__()
         self.repeat = repeat
-    def to_xml(self, indent = ""):
+    def to_xml(self, indent: str = "") -> str:
         s = indent + ('<sequent repeat="%d">\n' % self.repeat)
         s += super(StateLoop, self).to_xml(indent + "  ")
         s += indent + '</sequent>\n'
         return s
-    def __repr__(self):
+    def __repr__(self) -> str:
         return 'StateLoop(repeat=%d, %s)' \
             % (self.repeat, repr(self.list))
 
@@ -78,32 +78,32 @@ class Experiment:
     :param list rf_gates: list of rf gates ttl channels  len(rf_gates) == len(rf_sources)
 
     """
-    job_id = 0
+    job_id: int = 0
 
-    def __init__(self, gating=None, rf_sources=[], rf_gates=[]):
-        self.job_id = Experiment.job_id
+    def __init__(self, gating: float | None = None, rf_sources: list[int] = [], rf_gates: list[int] = []) -> None:
+        self.job_id: int = Experiment.job_id
         Experiment.job_id += 1
 
-        self.state_list = StateList()
-        self.list_stack = []
-        self.description = { }
-        self.gating = gating
+        self.state_list: StateList = StateList()
+        self.list_stack: list[StateList] = []
+        self.description: dict[str, object] = { }
+        self.gating: float | None = gating
         assert type(rf_sources) == type(list()), "rf_sources needs to be a list with the channels"
         assert type(rf_gates) == type(list()), "rf_gates needs to be a list with the channels"
         assert len(rf_gates) == len(rf_sources), "rf_sources and rf_gates must have equal number of entries"
-        self.rf_sources = rf_sources
-        self.rf_gates = rf_gates
+        self.rf_sources: list[int] = rf_sources
+        self.rf_gates: list[int] = rf_gates
         
         #for tracking the experiment length:
         #because loops are possible we need to track the length for each loop level
-        self.total_time=[0.0]
+        self.total_time: list[float] = [0.0]
         
         #and we need to know the number of iterations of the loops to multiply the state.
-        self.loop_iterations=[1]
+        self.loop_iterations: list[int] = [1]
 
     # Commands -------------------------------------------------------------------------------------
 
-    def ttl_pulse(self, length, channel = None, value = None):
+    def ttl_pulse(self, length: float, channel: int | None = None, value: int | None = None) -> None:
         """
         Creates a state with length **length** and switches requested bits of the pulse programmer to HIGH.
 
@@ -151,7 +151,7 @@ class Experiment:
         self.total_time[-1] += length
 
     ## Same as ttl_pulse, but no *channel* keyword
-    def ttls(self, length = None, value = None):
+    def ttls(self, length: float | None = None, value: int | None = None) -> None:
         """
         Same as ttl_pulse, but no *channel* keyword
 
@@ -159,6 +159,7 @@ class Experiment:
         :param int value: lines to set (integer)
         :return:
         """
+        assert value is not None, "ttls needs an integer value"
         the_value=int(value)
         s_content = '<ttlout value="0x%06x"/>' % the_value
         if length is not None:
@@ -168,7 +169,7 @@ class Experiment:
             self.state_list.append(s_content)
             
 
-    def rf_pulse(self, length=None, phase=0, source=0):
+    def rf_pulse(self, length: float | None = None, phase: float = 0, source: int = 0) -> None:
         """
         Make an rf pulse, including gating and phase switching. Only possible if Experiment is configured.
 
@@ -183,24 +184,24 @@ class Experiment:
         self.ttls(length=length, value=self.rf_gates[source]+self.rf_sources[source])
 
     ## Beginning of a new state
-    def state_start(self, time):
+    def state_start(self, time: float) -> None:
         """
         Starts a state in the pulse programs with duration *time*.
         This must be closed with :func:`state_end`.
 
         Typically only used in very special cases.
         """
-        self.state_list.append('<state time="%s">\n' % repr(time))
+        self.state_list.append('<state time="%s">\n' % repr(float(time)))
         self.total_time[-1] += time
 
     ## End of *state_start*
-    def state_end(self):
+    def state_end(self) -> None:
         """
         Closes a previous :func:`state_start`
         """
         self.state_list.append('</state>\n')
 
-    def wait(self, time, ttls=None, gating=False):
+    def wait(self, time: float, ttls: int | None = None, gating: bool = False) -> None:
         """
         Waits for a specified amount of time without performing any actions.
 
@@ -241,6 +242,7 @@ class Experiment:
 
         """
         if gating:
+            assert self.gating is not None, "gating time must be set for gating=True"
             time -= self.gating
         if ttls is not None:
             s_content = '<ttlout value="0x%06x"/>' % ttls
@@ -251,7 +253,7 @@ class Experiment:
         self.total_time[-1] += time
 
 
-    def record(self, samples, frequency, timelength=None, sensitivity = None, ttls=None, channels = 3, offset = None, impedance = None):
+    def record(self, samples: int, frequency: float, timelength: float | None = None, sensitivity: float | list[float] | None = None, ttls: int | None = None, channels: int = 3, offset: int | list[int] | None = None, impedance: float | list[float] | None = None) -> None:
         """
         Records data with a given number of samples, sampling frequency, and sensitivity.
         This command starts data acquisition from the ADC (Analog-to-Digital Converter).
@@ -365,7 +367,7 @@ class Experiment:
 
     ## Create a loop on the pulse programmer. Loop contents can not change inside the loop.
     # @params iterations Number of loop iterations
-    def loop_start(self, iterations):
+    def loop_start(self, iterations: int) -> None:
         """
         creates a loop of given number of iterations and has to be closed by loop_end().
         Commands inside the loop can not change, i.e., the parameters are the same for each loop run.
@@ -384,7 +386,7 @@ class Experiment:
         self.loop_iterations.append(iterations)
 
     ## End loop state
-    def loop_end(self):
+    def loop_end(self) -> None:
         """
         Closes a l:func:`loop_start` statement.
         """
@@ -396,23 +398,23 @@ class Experiment:
         self.total_time[-1] += looptime*loopiterations
 
 
-    def set_frequency(self, frequency, phase, ttls=0):
+    def set_frequency(self, frequency: float, phase: float, ttls: int = 0) -> None:
         """
         Sets the frequency and phase of the frequency source and optionally further channels.
         The state length is 2 µs. Stabilisation time depends on RF source.
 
         :param float frequency: frequency to set in Hz
-        :param float phase: phase to set
+        :param float phase: phase to set, normalized to the range [0, 360)
         :param int ttls: default=0, additional ttl lines to set (integer)
         """
-        s_content = '<analogout id="0" f="%f" phase="%f"/>' % (frequency, phase)
+        s_content = '<analogout id="0" f="%f" phase="%f"/>' % (frequency, phase % 360)
         if ttls != 0:
             s_content += '<ttlout value="0x%06x"/>' % ttls
         self.state_list.append(StateSimple(2e-6, s_content))
         
         self.total_time[-1] += 2e-6
 
-    def set_pfg(self, dac_value=None, length=None, shape=('rec', 0), trigger=4, is_seq=False):
+    def set_pfg(self, dac_value: int | None = None, length: float | None = None, shape: tuple[str, float] = ('rec', 0), trigger: int = 4, is_seq: bool = False) -> None:
         """
         This sets the value for the PFG, it also sets it back to dac_value=0 automatically.
         If you don't wish to do so (i.e. line shapes)  set is_seq=1
@@ -480,7 +482,7 @@ class Experiment:
         else: # don't know what to do
             raise SyntaxError("form is unknown: %s"%form)
 
-    def set_dac(self, dac_value, dac_id=1, length=None, is_seq=False, ttls=0):
+    def set_dac(self, dac_value: int, dac_id: int = 1, length: float | None = None, is_seq: bool = False, ttls: int = 0) -> None:
         """
         Sets the value for the DAC (Digital-to-Analog Converter) and optionally additional TTL lines.
         This command is used to control analog output signals, such as pulsed field gradients.
@@ -544,7 +546,7 @@ class Experiment:
             
             self.total_time[-1] += 42*9e-8
 
-    def set_phase(self, phase, ttls=0):
+    def set_phase(self, phase: float, ttls: int = 0) -> None:
         """
         Sets the phase of the RF source to the specified value.
 
@@ -580,8 +582,9 @@ class Experiment:
         - The state length for this command is 0.5 µs.
         - The stabilization time for the phase change is RF source dependent. A typical value for PTS310 is 2 µs.
         - The phase is relative to the initial phase set at start of the experiment.
+        - Negative phases are normalized to the range [0, 360) before writing the job file.
         """
-        s_content = '<analogout phase="%f" />' % (phase)
+        s_content = '<analogout phase="%f" />' % (phase % 360)
         if ttls!=0:
             s_content += '<ttlout value="%d"/>' % ttls
         self.state_list.append(StateSimple(0.5e-6, s_content))
@@ -589,7 +592,7 @@ class Experiment:
         self.total_time[-1] += 0.5e-6
 
 
-    def set_description(self, key, value):
+    def set_description(self, key: str, value: object) -> None:
         """
         Sets a description key-value pair in the experiment description dictionary.
 
@@ -629,7 +632,7 @@ class Experiment:
             print('Warning: Overwriting existing description "%s" = "%s" with "%s"' % (key, self.description[key], value))
         self.description[key] = value
 
-    def set_pts_local(self):
+    def set_pts_local(self) -> None:
         """
         Sets the PTS310/PTS500 frequency source to local mode.
         State length is 2 µs.
@@ -653,7 +656,7 @@ class Experiment:
         return self.job_id
 
 
-    def write_xml_string(self):
+    def write_xml_string(self) -> str:
         """
         Returns the current program as a string
 
@@ -678,19 +681,19 @@ class Experiment:
                     value=""
                 elif type(value) is float or isinstance(value, numpy.floating):
                     type_string="Float"
-                    value=repr(value)
+                    value=repr(float(value))
                 elif type(value) is int or isinstance(value, numpy.integer):
                     type_string="Int"
-                    value=repr(value)
+                    value=repr(int(value))
                 elif type(value) is int:
                     type_string="Long"
                     value=repr(value)
                 elif type(value) is complex or isinstance(value, numpy.complexfloating):
                     type_string="Complex"
-                    value=repr(value)
+                    value=repr(complex(value))
                 elif type(value) is bool or isinstance(value, numpy.bool_):
                     type_string="Boolean"
-                    value=repr(value)
+                    value=repr(bool(value))
                 elif type(value) in (str,):
                     type_string="String"
                 else:
@@ -706,14 +709,14 @@ class Experiment:
 
         return xml_string
 
-    def write_quit_job(self):
+    def write_quit_job(self) -> str:
         """
         Returns a xml quit-job
         :returns: XML quit-job
         """
         return '<?xml version="1.0" encoding="ISO-8859-1"?>\n<quit/>'
     
-    def get_length(self):
+    def get_length(self) -> float:
         timelength = 0.0
         #calculate the correct timelength also for unclosed loops, if there is no unclosed loop
         #the timelength of this experiment is self.total_time[-1].
@@ -723,13 +726,13 @@ class Experiment:
         return timelength
 
 class Quit(Experiment):
-    def write_xml_string(self):
+    def write_xml_string(self) -> str:
         return '<?xml version="1.0" encoding="ISO-8859-1"?>\n<quit no="%d"/>'%self.job_id
 
 
 # /Public Methods ------------------------------------------------------------------------------
 
-def self_test():
+def self_test() -> None:
     e = Experiment()
     e.set_description("key", "value")
     e.set_frequency(85e6, 90, ttls=16)

src/damaris/experiments/dac.py

@@ -2,7 +2,7 @@
 """
 This module holds everything connected with the DAC and PFG
 """
-def conv(I_out=0):
+def conv(I_out: float = 0) -> int:
     """
     converts the demanded Output current in Integer
     """

src/damaris/gui/BackendDriver.py

@@ -5,12 +5,16 @@ import sys
 import time
 import re
 import glob
+import logging
 from . import ExperimentWriter
 from . import ResultReader
+from .logging_utils import FileTailerHandler
 import threading
 import types
 import signal
 
+logger = logging.getLogger("damaris.backend_driver")
+
 if sys.platform=="win32":
     import winreg
 
@@ -40,7 +44,7 @@ class BackendDriver(threading.Thread):
             try:
                 os.makedirs(os.path.abspath(self.spool_dir))
             except OSError as e:
-                print(e)
+                logger.error("Could not create backend's spool directory %s: %s", self.spool_dir, e)
                 raise AssertionError("could not create backend's spool directory %s "%self.spool_dir)
         
         # remove stale state filenames
@@ -62,10 +66,10 @@ class BackendDriver(threading.Thread):
                     if os.path.isdir("/proc/%d"%core_pid):
                         raise AssertionError("found backend with pid %d (state file %s) in same spool dir"%(core_pid,statefilename))
                     else:
-                        print("removing stale backend state file", statefilename)
+                        logger.warning("Removing stale backend state file: %s", statefilename)
                         os.remove(statefilename)
         else:
-            print("todo: take care of existing backend state files")
+            logger.warning("TODO: take care of existing backend state files (Windows)")
 
         self.result_reader = ResultReader.BlockingResultReader(self.spool_dir,
                                                                no=0,
@@ -81,21 +85,9 @@ class BackendDriver(threading.Thread):
         self.raised_exception=None
 
     def run(self):
-        # take care of older logfiles
-        self.core_output_filename=os.path.join(self.spool_dir,"logdata")
-        if os.path.isfile(self.core_output_filename):
-            i=0
-            max_logs=100
-            while os.path.isfile(self.core_output_filename+".%02d"%i):
-                i+=1
-            while (i>=max_logs):
-                i-=1
-                os.remove(self.core_output_filename+".%02d"%i)
-            for j in range(i):
-                os.rename(self.core_output_filename+".%02d"%(i-j-1),self.core_output_filename+".%02d"%(i-j))
-            os.rename(self.core_output_filename, self.core_output_filename+".%02d"%0)
-        # create logfile
-        self.core_output=open(self.core_output_filename,"w")
+        # Log file for backend output
+        self.core_output_filename = os.path.join(self.spool_dir, "logdata")
+        self.core_output = open(self.core_output_filename, "w")
 
         # again look out for existing state files
         state_files=glob.glob(os.path.join(self.spool_dir,"*.state"))
@@ -118,27 +110,28 @@ class BackendDriver(threading.Thread):
                                              stdout=self.core_output,
                                              stderr=self.core_output)
 
+        # Start tailing backend log file for unified logging
+        self._log_tailer = FileTailerHandler(self.core_output_filename)
+        self._log_tailer.start()
+
         # wait till state file shows up
         timeout=10
         # to do: how should I know core's state name????!!!!!
         self.statefilename=None
         state_files=glob.glob(os.path.join(self.spool_dir,"*.state"))
         while len(state_files)==0:
-            if timeout<0 or self.core_input is None or self.core_input.poll() is not None or self.quit_flag.isSet():
+            if timeout<0 or self.core_input is None or self.core_input.poll() is not None or self.quit_flag.is_set():
                 # look into core log file and include contents
-                print(timeout, self.core_input, self.quit_flag.isSet())
-                if self.core_input is not None:
-                    print(self.core_input.poll())
+                logger.error("Backend failed to start: timeout=%s poll=%s", timeout, self.core_input.poll())
                 log_message=''
                 self.core_input=None
                 if os.path.isfile(self.core_output_filename):
-                    # to do include log data
                     log_message='\n'+''.join(open(self.core_output_filename,"r", errors='replace').readlines()[:10])
                     if not log_message:
                         log_message=" no error message from core"
                 self.core_output.close()
                 self.raised_exception="no state file appeared or backend died away:"+log_message
-                print(self.raised_exception)
+                logger.error(self.raised_exception)
                 self.quit_flag.set()
                 return
             time.sleep(0.05)
@@ -147,7 +140,7 @@ class BackendDriver(threading.Thread):
 
         # save the one
         if len(state_files)>1:
-            print("did find more than one state file, taking first one!")
+            logger.warning("Found more than one state file, taking first one!")
         self.statefilename=state_files[0]
 
         # read state file
@@ -163,10 +156,10 @@ class BackendDriver(threading.Thread):
                 break
 
         # wait on flag and look after backend
-        while not self.quit_flag.isSet() and self.is_busy():
+        while not self.quit_flag.is_set() and self.is_busy():
             self.quit_flag.wait(0.1)
 
-        if self.quit_flag.isSet():
+        if self.quit_flag.is_set():
             self.stop_queue()
             while self.is_busy():
                 time.sleep(0.1)
@@ -182,28 +175,28 @@ class BackendDriver(threading.Thread):
                     backend_result=self.core_input.poll()
                     if backend_result is not None: break
                     if wait_loop_counter==10:
-                        print("sending termination signal to backend process")
+                        logger.warning("Sending termination signal to backend process")
                         self.send_signal("SIGTERM")
                     elif wait_loop_counter==20:
-                        print("sending kill signal to backend process")
+                        logger.warning("Sending kill signal to backend process")
                         self.send_signal("SIGKILL")
                     elif wait_loop_counter>30:
-                        print("no longer waiting for backend shutdown")
+                        logger.warning("No longer waiting for backend shutdown")
                         break
 
                 if backend_result is None:
-                    print("backend dit not end properly, please stop it manually")
+                    logger.error("Backend did not end properly, please stop it manually")
                 elif backend_result>0:
-                    print("backend returned ", backend_result)
+                    logger.error("Backend returned exit code %d", backend_result)
                 elif backend_result<0:
                     sig_name=[x for x in dir(signal) if x.startswith("SIG") and \
                                     x[3]!="_" and \
                                     (type(signal.__dict__[x])is int) and \
                                     signal.__dict__[x]==-backend_result]
                     if sig_name:
-                        print("backend was terminated by signal ",sig_name[0])
+                        logger.error("Backend was terminated by signal %s", sig_name[0])
                     else:
-                        print("backend was terminated by signal no",-backend_result)
+                        logger.error("Backend was terminated by signal no %d", -backend_result)
             self.core_input = None
             self.core_pid = None
 
@@ -226,12 +219,6 @@ class BackendDriver(threading.Thread):
         if os.path.isfile(resultfilename):
             os.remove(resultfilename)
 
-    def get_messages(self):
-        # return pending messages
-        if self.core_output.tell()==os.path.getsize(self.core_output_filename):
-            return None
-        return self.core_output.read()
-
     def restart_queue(self):
         self.send_signal("SIGUSR1")
 
@@ -250,7 +237,7 @@ class BackendDriver(threading.Thread):
 
     def send_signal(self, sig):
         if self.core_pid is None:
-            print("BackendDriver.send_signal is called with core_pid=None")
+            logger.error("BackendDriver.send_signal called with core_pid=None")
             return
         try:
             if sys.platform[:5]=="linux":
@@ -262,7 +249,7 @@ class BackendDriver(threading.Thread):
                 kill_command=os.path.join(cygwin_path,"bin","kill.exe")
                 os.popen("%s -%s %d"%(kill_command,sig,self.core_pid))
         except OSError as e:
-            print("could not send signal %s to core: %s"%(sig, str(e)))
+            logger.error("Could not send signal %s to core: %s", sig, str(e))
             
 
     def is_busy(self):
@@ -294,3 +281,5 @@ class BackendDriver(threading.Thread):
         if self.core_output:
             self.core_output.close()
             self.core_output=None
+        if hasattr(self, '_log_tailer'):
+            self._log_tailer.stop()

src/damaris/gui/ExperimentHandling.py

@@ -3,39 +3,88 @@ import io
 import traceback
 import sys
 import time
+import logging
 import ast
+import contextlib
 from damaris.experiments.Experiment import Quit
 from damaris.experiments import Experiment
 
+logger = logging.getLogger("damaris.experiment_handling")
+
 class StopExperiment(Exception):
     pass
 
+class _SandboxStdout:
+    """Redirects print() from sandboxed scripts to the GUI textview."""
+    def __init__(self, callback):
+        self._callback = callback
+        self._buffer = ""
+
+    def write(self, message):
+        self._buffer += message
+        if "\n" in self._buffer:
+            lines = self._buffer.split("\n")
+            self._buffer = lines[-1]
+            for line in lines[:-1]:
+                self._callback(line + "\n")
+
+    def flush(self):
+        if self._buffer:
+            self._callback(self._buffer)
+            self._buffer = ""
+
+    def isatty(self):
+        return False
+
+
+@contextlib.contextmanager
+def _sandbox_stdout_redirect(log_callback):
+    """Temporarily redirect process-global stdout/stderr to the sandbox log."""
+    old_stdout, old_stderr = sys.stdout, sys.stderr
+    sandbox = _SandboxStdout(log_callback)
+    try:
+        sys.stdout = sandbox
+        sys.stderr = sandbox
+        yield sandbox
+    finally:
+        sys.stdout, sys.stderr = old_stdout, old_stderr
+
+
 class ExperimentHandling(threading.Thread):
     """
     runs the experiment script in sandbox
     """
 
-    def __init__(self, script, exp_writer, data):
+    def __init__(self, script, exp_writer, data, log_callback=None):
         threading.Thread.__init__(self, name="experiment handler")
         self.script=script
         self.writer=exp_writer
         self.data=data
         self.quit_flag = threading.Event()
+        self._log_callback = log_callback
         if self.data is not None:
             self.data["__recentexperiment"]=-1
 
     def synchronize(self, before=0, waitsteps=0.1):
-        while (self.data["__recentexperiment"]>self.data["__recentresult"]+before) and not self.quit_flag.isSet():
+        while (self.data["__recentexperiment"]>self.data["__recentresult"]+before) and not self.quit_flag.is_set():
             self.quit_flag.wait(waitsteps)
-        if self.quit_flag.isSet():
+        if self.quit_flag.is_set():
             raise StopExperiment
         
     def sleep(self, seconds):
         self.quit_flag.wait(seconds)
-        if self.quit_flag.isSet():
+        if self.quit_flag.is_set():
             raise StopExperiment
 
     def run(self):
+        # Redirect stdout/stderr so print() in sandboxed scripts goes to the GUI
+        if self._log_callback is not None:
+            with _sandbox_stdout_redirect(self._log_callback):
+                self._run_script()
+        else:
+            self._run_script()
+
+    def _run_script(self):
         dataspace={}
         exp_classes = __import__('damaris.experiments', dataspace, dataspace, ['Experiment'])
         for name in dir(exp_classes):
@@ -66,7 +115,7 @@ class ExperimentHandling(threading.Thread):
                             found_blocking_sleep = True
                 
                 if found_blocking_sleep:
-                    print("Warning: time.sleep() detected in experiment script. This is blocking and not interruptible. Please use sleep() instead.")
+                    logger.warning("time.sleep() detected in experiment script. This is blocking and not interruptible. Please use sleep() instead.")
                     break
         except Exception:
             pass
@@ -97,7 +146,7 @@ class ExperimentHandling(threading.Thread):
                 traceback_file=None
                 return
         
-        while exp_iterator is not None and not self.quit_flag.isSet():
+        while exp_iterator is not None and not self.quit_flag.is_set():
             # get next experiment from script 
             try:
                 job=next(exp_iterator)
@@ -137,7 +186,7 @@ class ExperimentHandling(threading.Thread):
                     # relax for a short time
                     if "__resultsinadvance" in self.data and self.data["__resultsinadvance"]+100<job.job_id:
                         self.quit_flag.wait(0.05)
-                if self.quit_flag.isSet():
+                if self.quit_flag.is_set():
                     dataspace=None
                     exp_iterator=None
                     break

src/damaris/gui/ResultHandling.py

@@ -4,25 +4,37 @@ import sys
 import os
 import os.path
 import traceback
+import logging
 import ast
 from damaris.data import Resultable
-from damaris.gui.ExperimentHandling import StopExperiment
+from damaris.gui.ExperimentHandling import StopExperiment, _sandbox_stdout_redirect
+
+logger = logging.getLogger("damaris.result_handling")
 
 class ResultHandling(threading.Thread):
     """
     runs the result script in sandbox
     """
 
-    def __init__(self, script_data, result_iterator, data_pool):
+    def __init__(self, script_data, result_iterator, data_pool, log_callback=None):
         threading.Thread.__init__(self,name="result handler")
         self.script=script_data
         self.results=result_iterator
         self.data_space=data_pool
         self.quit_flag=self.results.quit_flag
+        self._log_callback = log_callback
         if self.data_space is not None:
             self.data_space["__recentresult"]=-1
 
     def run(self):
+        # Redirect stdout/stderr so print() in sandboxed scripts goes to the GUI
+        if self._log_callback is not None:
+            with _sandbox_stdout_redirect(self._log_callback):
+                self._run_script()
+        else:
+            self._run_script()
+
+    def _run_script(self):
         # execute it
         dataspace={}
         data_classes = __import__('damaris.data', dataspace, dataspace, ['*'])
@@ -52,7 +64,7 @@ class ResultHandling(threading.Thread):
                             found_blocking_sleep = True
                 
                 if found_blocking_sleep:
-                    print("Warning: time.sleep() detected in result script. This is blocking and not interruptible. Please use sleep() instead.")
+                    logger.warning("time.sleep() detected in result script. This is blocking and not interruptible. Please use sleep() instead.")
                     break
         except Exception:
             pass
@@ -87,24 +99,24 @@ class ResultHandling(threading.Thread):
 
     def sleep(self, seconds):
         self.quit_flag.wait(seconds)
-        if self.quit_flag.isSet():
+        if self.quit_flag.is_set():
             raise StopExperiment
 
     def __iter__(self):
-        if self.quit_flag.isSet():
+        if self.quit_flag.is_set():
             self.results=None
             return
         for i in self.results:
             if hasattr(self.results, "in_advance"):
                 self.data_space["__resultsinadvance"]=self.results.in_advance
-            if self.quit_flag.isSet():
+            if self.quit_flag.is_set():
                 self.results=None
                 return
             if isinstance(i, Resultable.Resultable):
                 if self.data_space is not None:
                     self.data_space["__recentresult"]=i.job_id+0
             yield i
-            if self.quit_flag.isSet():
+            if self.quit_flag.is_set():
                 self.results=None
                 return
 

src/damaris/gui/ResultReader.py

@@ -15,7 +15,7 @@ import base64
 import numpy
 
 try:
-    import xml.etree.cElementTree
+    import xml.etree.ElementTree
     ELEMENT_TREE = True
 except:
     import xml.parsers.expat
@@ -109,7 +109,7 @@ class ResultReader:
         self.result_job_number = None
         # Job Date is set in __read_file()
         self.__filetype = None
-        for elem in xml.etree.cElementTree.ElementTree(file=in_file).iter():
+        for elem in xml.etree.ElementTree.ElementTree(file=in_file).iter():
             if elem.tag == 'result':
                 self.result_job_number = int(elem.get("job"))
                 pass
@@ -135,7 +135,8 @@ class ResultReader:
                     elif t == "Complex":
                         self.result_description[k]=complex(v)
                     elif t == "Boolean":
-                        self.result_description[k]=bool(v)
+                        # bool("False") would be True, compare the text instead
+                        self.result_description[k]=(v == "True")
                     elif t == "String":
                         self.result_description[k]=v
                     else:
@@ -231,7 +232,6 @@ class ResultReader:
                 for i in range(nChannels):
                     # split interleaved data
                     self.result.y[i][tmp_index:tmp_index+tmp_size] = tmp_part[i::nChannels]
-                    self.result.y[i][tmp_index:tmp_index+tmp_size] = tmp_part[i::nChannels]
 
                 if self.result.index != []:
                     self.result.index.append((tmp_index, tmp_index+tmp_size-1))
@@ -503,7 +503,8 @@ class ResultReader:
             elif t == "Complex":
                 self.result_description[k]=complex(v)
             elif t == "Boolean":
-                self.result_description[k]=bool(v)
+                # bool("False") would be True, compare the text instead
+                self.result_description[k]=(v == "True")
             elif t == "String":
                 self.result_description[k]=v
             else:
@@ -534,7 +535,7 @@ class BlockingResultReader(ResultReader):
     def __init__(self, spool_dir=".", no=0, result_pattern="job.%09d.result", clear_jobs=False, clear_results=False):
         ResultReader.__init__(self, spool_dir, no, result_pattern, clear_jobs=clear_jobs, clear_results=clear_results)
         self.stop_no=None # end of job queue
-        self.poll_time=0.1 # sleep interval for polling results, <0 means no polling and stop
+        self.poll_time=0.05 # sleep interval for polling results, <0 means no polling and stop
         self.in_advance=0
 
     def __iter__(self):
@@ -543,7 +544,7 @@ class BlockingResultReader(ResultReader):
         block until result is available
         """
         expected_filename=os.path.join(self.spool_dir,self.result_pattern%(self.no))
-        while (not self.quit_flag.isSet()) and (self.stop_no is None or self.stop_no>self.no):
+        while (not self.quit_flag.is_set()) and (self.stop_no is None or self.stop_no>self.no):
             if not os.access(expected_filename,os.R_OK):
                 # stop polling, if required
                 if self.poll_time<0: break
@@ -559,11 +560,11 @@ class BlockingResultReader(ResultReader):
                 self.in_advance+=1
                 in_advance_filename=os.path.join(self.spool_dir,self.result_pattern%(self.in_advance+1))
 
-            if self.quit_flag.isSet(): break
+            if self.quit_flag.is_set(): break
             r=self.get_result_object(expected_filename)
-            if self.quit_flag.isSet(): break
+            if self.quit_flag.is_set(): break
             
-            if self.quit_flag.isSet(): break
+            if self.quit_flag.is_set(): break
             yield r
             
             if self.clear_results:

src/damaris/gui/TemperatureMonitor.py

@@ -13,25 +13,6 @@ import datetime
 from damaris.data import TemperatureResult
 from damaris.tools.temperature import TemperatureController
 
-# GTK imports are done lazily to avoid initialization issues
-_HAS_GTK = None
-_gdk = None
-
-
-def _get_gdk():
-    """Lazily import gdk to avoid GTK initialization issues."""
-    global _HAS_GTK, _gdk
-    if _HAS_GTK is None:
-        try:
-            import gi
-            gi.require_version('Gdk', '3.0')
-            from gi.repository import Gdk as gdk
-            _gdk = gdk
-            _HAS_GTK = True
-        except (ImportError, ValueError):
-            _HAS_GTK = False
-    return _gdk if _HAS_GTK else None
-
 
 class TemperatureMonitor(threading.Thread):
     """
@@ -72,6 +53,10 @@ class TemperatureMonitor(threading.Thread):
         self.result = TemperatureResult()
         self._running = False
         self._lock = threading.RLock()
+        self._consecutive_failures = 0
+        self._max_consecutive_failures = 10
+        self._last_error = None
+        self._connection_ok = True
         
         # Initialize the result with metadata
         self.result.xlabel = "Time (s)"
@@ -124,21 +109,29 @@ class TemperatureMonitor(threading.Thread):
                         data_pool_result.job_id = self.result.job_id if hasattr(self.result, 'job_id') else None
                         data_pool_result.job_date = self.result.job_date if hasattr(self.result, 'job_date') else None
                     
-                    # Store in data_pool in GTK thread (if GTK available)
-                    gdk = _get_gdk()
-                    if gdk is not None:
-                        gdk.threads_enter()
-                        try:
-                            self.data_pool[self.data_key] = data_pool_result
-                        finally:
-                            gdk.threads_leave()
-                    else:
-                        self.data_pool[self.data_key] = data_pool_result
+                    # Store in data_pool (dict item assignment is atomic under the GIL)
+                    self.data_pool[self.data_key] = data_pool_result
+
+                # Reset consecutive failure counter on successful read
+                self._consecutive_failures = 0
+                self._last_error = None
+                self._connection_ok = True
                 
             except Exception as e:
-                print(f"TemperatureMonitor error: {e}")
-                import traceback
-                traceback.print_exc()
+                self._consecutive_failures += 1
+                self._last_error = str(e)
+                if self._consecutive_failures <= 3:
+                    print(f"TemperatureMonitor read error (attempt {self._consecutive_failures}): {e}")
+                elif self._consecutive_failures == 5:
+                    print(f"TemperatureMonitor: repeated read errors, checking connection...")
+                    if hasattr(self.controller, 'check_connection'):
+                        self._connection_ok = self.controller.check_connection()
+                    if not self._connection_ok:
+                        print(f"TemperatureMonitor: WARNING - controller appears disconnected after {self._consecutive_failures} failures")
+                elif self._consecutive_failures >= self._max_consecutive_failures:
+                    print(f"TemperatureMonitor: ERROR - {self._consecutive_failures} consecutive failures, controller may be disconnected")
+                else:
+                    print(f"TemperatureMonitor error (failure #{self._consecutive_failures}): {e}")
             
             # Sleep in small increments for responsive shutdown
             sleep_remaining = self.interval

src/damaris/gui/__init__.py

@@ -0,0 +1,5 @@
+from .logging_utils import (
+    FileTailerHandler,
+    LogDispatcher,
+    setup_logging,
+)

src/damaris/gui/damaris3.glade

@@ -91,18 +91,21 @@
     </data>
   </object>
   <object class="GtkTextBuffer" id="textbuffer1">
-    <property name="text">DAMARIS python frontend version %(damarisversion)s, 2012
+    <property name="text">DAMARIS (DArmstadt Magnetic Resonance Instrument Software)
+Python frontend version %(damarisversion)s
+
+Copyright © 2004-2026 Achim Gädke, Markus Rosenstihl, and others.
+Homepage: https://gitea.pkm.physik.tu-darmstadt.de/IPKM/python3-damaris
+
+Components version information:
 %(moduleversions)s
 
-DArmstadt Magnetic Resonance Instrument Software, 2004-2012
-Homepage https://element.fkp.physik.tu-darmstadt.de/damaris_cms/
-
-Python frontend written by Christopher Schmitt, Achim Gädke, Markus Rosenstihl and Christian Tacke
-Backends written by Achim Gädke, Markus Rosenstihl, Christian Tacke, tested by Holger Stork
 The drivers included in the backend may use proprietary interfaces.
 The source code distribution may include commercial third party's code,
 which is specially marked and not covered by this license.
 
+-------------------------------------------------------------------------------
+
    GNU GENERAL PUBLIC LICENSE
 		       Version 2, June 1991
 
@@ -641,6 +644,21 @@ Public License instead of this License.
                 <property name="homogeneous">True</property>
               </packing>
             </child>
+            <child>
+              <object class="GtkToolButton" id="toolbar_estimate_button">
+                <property name="visible">True</property>
+                <property name="can-focus">False</property>
+                <property name="tooltip-text" translatable="yes">Estimate experiment duration</property>
+                <property name="label">Estimate</property>
+                <property name="use-underline">True</property>
+                <property name="icon-name">dialog-information</property>
+                <signal name="clicked" handler="on_toolbar_estimate_button_clicked" swapped="no"/>
+              </object>
+              <packing>
+                <property name="expand">False</property>
+                <property name="homogeneous">True</property>
+              </packing>
+            </child>
             <child>
               <object class="GtkToolItem" id="toolitem4">
                 <property name="visible">True</property>
@@ -909,7 +927,6 @@ Public License instead of this License.
                       <object class="GtkSourceView" id="data_handling_textview">
                         <property name="visible">True</property>
                         <property name="can-focus">True</property>
-                        <signal name="event" handler="on_data_handling_textview_event" swapped="no"/>
                       </object>
                     </child>
                   </object>

src/damaris/gui/logging_utils.py

@@ -0,0 +1,256 @@
+# -*- coding: utf-8 -*-
+"""
+Unified logging for DAMARIS.
+
+Provides:
+- FileTailerHandler: reads new lines from the backend log file and emits them
+  as logging records into the damaris.backend logger.
+- LogDispatcher: a logging.Handler that dispatches records to both a GTK
+  messages_textview (via idle_add) and the console.
+- setup_logging(): convenience function to configure the root logger.
+"""
+
+import logging
+import os
+import queue
+import re
+import sys
+import threading
+import time
+
+# ---------------------------------------------------------------------------
+# Severity prefix pattern for C++ backend messages
+# ---------------------------------------------------------------------------
+_BACKEND_PREFIX_RE = re.compile(r"^\[(DEBUG|INFO|WARNING|ERROR|CRITICAL)\]\s*(.*)")
+
+# Mapping from string prefix to logging level
+_PREFIX_TO_LEVEL = {
+    "DEBUG": logging.DEBUG,
+    "INFO": logging.INFO,
+    "WARNING": logging.WARNING,
+    "ERROR": logging.ERROR,
+    "CRITICAL": logging.CRITICAL,
+}
+
+
+# ---------------------------------------------------------------------------
+# FileTailerHandler
+# ---------------------------------------------------------------------------
+class FileTailerHandler(logging.Handler):
+    """
+    Reads new lines from a log file and emits them as log records.
+
+    Runs a background thread that polls the file every `poll_interval` seconds.
+    New lines are parsed for a severity prefix (e.g. ``[INFO] message``) and
+    emitted as LogRecords under the ``damaris.backend`` logger.  Lines without
+    a recognized prefix are emitted at INFO level.
+
+    Parameters
+    ----------
+    filename : str
+        Path to the log file to tail.
+    poll_interval : float, optional
+        Seconds between polls (default 0.1).
+    logger_name : str, optional
+        Logger name for emitted records (default "damaris.backend").
+    """
+
+    def __init__(self, filename, poll_interval=0.1, logger_name="damaris.backend"):
+        super().__init__()
+        self._filename = filename
+        self._poll_interval = poll_interval
+        self._logger_name = logger_name
+        self._queue = queue.Queue()
+        self._thread = None
+        self._stop_event = threading.Event()
+        self._position = 0
+        self._file = None
+
+    def _tailer_loop(self):
+        """Background thread: open file, read new lines, push to queue."""
+        while not self._stop_event.is_set():
+            try:
+                if not os.path.isfile(self._filename):
+                    time.sleep(self._poll_interval)
+                    continue
+
+                fh = open(self._filename, "r", errors="replace")
+                fh.seek(self._position)
+                new_lines = fh.readlines()
+                self._position = fh.tell()
+                fh.close()
+
+                for line in new_lines:
+                    line = line.rstrip("\n\r")
+                    if line:
+                        self._queue.put(line)
+
+            except Exception:
+                # File may have been rotated or deleted; reset position
+                self._position = 0
+
+            self._stop_event.wait(self._poll_interval)
+
+    def emit(self, record):
+        """Called when a log record is ready.  Not used directly — lines are
+        pushed from the queue by _process_queue()."""
+        pass
+
+    def _process_queue(self):
+        """Drain the queue and emit parsed log records.  Called from the
+        thread that owns the handler (typically the GUI thread)."""
+        while not self._queue.empty():
+            try:
+                line = self._queue.get_nowait()
+            except queue.Empty:
+                break
+
+            m = _BACKEND_PREFIX_RE.match(line)
+            if m:
+                level_str, message = m.group(1), m.group(2)
+                level = _PREFIX_TO_LEVEL.get(level_str, logging.INFO)
+            else:
+                level = logging.INFO
+                message = line
+
+            # Create a LogRecord from the raw message
+            record = logging.LogRecord(
+                name=self._logger_name,
+                level=level,
+                pathname="(tailer)",
+                lineno=0,
+                msg=message,
+                args=None,
+                exc_info=None,
+            )
+            self.handle(record)
+
+    def start(self):
+        """Start the background tailing thread."""
+        if self._thread is not None and self._thread.is_alive():
+            return
+        self._stop_event.clear()
+        self._thread = threading.Thread(target=self._tailer_loop, name="log-tailer", daemon=True)
+        self._thread.start()
+
+    def stop(self):
+        """Stop the background thread."""
+        self._stop_event.set()
+        if self._thread is not None:
+            self._thread.join(timeout=2)
+            self._thread = None
+
+    def close(self):
+        self.stop()
+        super().close()
+
+
+# ---------------------------------------------------------------------------
+# LogDispatcher
+# ---------------------------------------------------------------------------
+class LogDispatcher(logging.Handler):
+    """
+    Dispatches log records to a GTK messages_textview and/or the console.
+
+    Parameters
+    ----------
+    gui_callback : callable or None
+        If not None, called with the formatted message string.  Must be
+        thread-safe (e.g. wrapped in ``gobject.idle_add``).  This is the
+        textview's ``__call__`` method from DamarisGUI.
+    console : bool, optional
+        If True, also emit to sys.stderr (default True).
+    """
+
+    def __init__(self, gui_callback=None, console=True):
+        super().__init__()
+        self._gui_callback = gui_callback
+        self._console_handler = logging.StreamHandler(sys.stderr) if console else None
+        if self._console_handler:
+            self._console_handler.setFormatter(
+                logging.Formatter("%(asctime)s [%(levelname)-8s] %(name)s: %(message)s",
+                                  datefmt="%H:%M:%S")
+            )
+
+    def emit(self, record):
+        try:
+            msg = self.format(record)
+
+            if self._gui_callback is not None:
+                try:
+                    import gi
+                    gi.require_version("Gdk", "3.0")
+                    from gi.repository import GObject as gobject
+                    gobject.idle_add(self._gui_callback, msg + "\n", priority=gobject.PRIORITY_LOW)
+                except Exception:
+                    # GUI not available or import failed — ignore
+                    pass
+
+            if self._console_handler is not None:
+                self._console_handler.emit(record)
+        except Exception:
+            self.handleError(record)
+
+    def close(self):
+        if self._console_handler:
+            self._console_handler.close()
+        super().close()
+
+
+# ---------------------------------------------------------------------------
+# setup_logging
+# ---------------------------------------------------------------------------
+def setup_logging(level=logging.INFO, gui_callback=None, backend_logfile=None):
+    """
+    Configure the DAMARIS logging system.
+
+    Parameters
+    ----------
+    level : int
+        Minimum logging level (default INFO).
+    gui_callback : callable or None
+        Optional callback for GUI textview output.
+    backend_logfile : str or None
+        Path to the backend log file.  If provided, a FileTailerHandler is
+        attached to the ``damaris.backend`` logger.
+
+    Returns
+    -------
+    logger : logging.Logger
+        The root damaris logger.
+    """
+    root = logging.getLogger("damaris")
+    root.setLevel(level)
+
+    # Remove existing handlers to avoid duplicates on re-init
+    root.handlers.clear()
+
+    # Console handler on the root logger
+    console = logging.StreamHandler(sys.stderr)
+    console.setFormatter(
+        logging.Formatter("%(asctime)s [%(levelname)-8s] %(name)s: %(message)s",
+                          datefmt="%H:%M:%S")
+    )
+    root.addHandler(console)
+
+    # GUI dispatcher
+    if gui_callback is not None:
+        dispatcher = LogDispatcher(gui_callback=gui_callback, console=False)
+        dispatcher.setFormatter(
+            logging.Formatter("[%(levelname)s] %(name)s: %(message)s")
+        )
+        root.addHandler(dispatcher)
+
+    # Backend file tailer
+    if backend_logfile is not None and os.path.isfile(backend_logfile):
+        tailer = FileTailerHandler(backend_logfile)
+        tailer.setFormatter(
+            logging.Formatter("[%(levelname)s] %(name)s: %(message)s")
+        )
+        backend_logger = logging.getLogger("damaris.backend")
+        backend_logger.setLevel(logging.DEBUG)
+        backend_logger.addHandler(tailer)
+        backend_logger.propagate = True  # bubble up to damaris root
+        tailer.start()
+
+    return root

src/damaris/gui/python.xml

@@ -191,6 +191,7 @@
         <keyword>set_pts_local</keyword>
         <keyword>wait</keyword>
         <keyword>record</keyword>
+        <keyword>lowpass</keyword>
     </keywordlist>
 
     <keywordlist style="datatype">

src/damaris/tools/eurotherm.py

@@ -1,5 +1,6 @@
 import serial
 import re
+import time
 import operator
 from functools import reduce
 
@@ -57,21 +58,22 @@ class Eurotherm(TemperatureController):
 	def send_read_param(self, param):
 		self.s.write(EOT + self.device + param + ENQ)
 
-	def read_param(self, param):
-		self.s.flushInput()
-		self.send_read_param(param)
-		answer = self.s.read(self._expect_len)
-		m = reply_pattern.search(answer)
-		if m is None:
-			# Reading _expect_len bytes was not enough...
-			answer += self.s.read(200)
+	def read_param(self, param, retries=3, retry_delay=0.05):
+		for attempt in range(retries):
+			self.s.flushInput()
+			self.send_read_param(param)
+			answer = self.s.read(self._expect_len)
 			m = reply_pattern.search(answer)
-		if m is not None:
-			self._expect_len = len(answer)
-			return m.group(1)
-		else:
-			print("received:", repr(answer))
-			return None
+			if m is None:
+				answer += self.s.read(200)
+				m = reply_pattern.search(answer)
+			if m is not None:
+				self._expect_len = len(answer)
+				return m.group(1)
+			if attempt < retries - 1:
+				time.sleep(retry_delay)
+		print("received:", repr(answer))
+		return None
 
 	def write_param(self, mnemonic, data):
 		if len(mnemonic) > 2:
@@ -128,6 +130,14 @@ class Eurotherm(TemperatureController):
 		"""Set target temperature in Celsius."""
 		return self.write_param('SL', str(temperature))
 
+	def check_connection(self) -> bool:
+		"""Check if the Eurotherm device is connected and responsive."""
+		try:
+			result = self.read_param('PV')
+			return result is not None
+		except Exception:
+			return False
+
 	def close(self):
 		"""Close the serial connection."""
 		if hasattr(self, 's') and self.s is not None:

src/damaris/tools/lakeshore.py

@@ -1,5 +1,6 @@
 import serial
 import re
+import time
 import numpy as np
 
 from .temperature import TemperatureController, register_controller
@@ -91,13 +92,17 @@ class Lakeshore(TemperatureController):
         else:
             return 0.0
 
-    def read_param(self, param):
+    def read_param(self, param, retries=3, retry_delay=0.05):
         """Read a parameter using simple ASCII commands."""
-        self.s.write(f"{param}?\n")
-        answer = self.s.readline().strip()
-        if answer and len(answer) > 1:
-            # Remove the first character (command echo) and return the value
-            return answer[1:]
+        for attempt in range(retries):
+            self.s.flushInput()
+            self.s.write(f"{param}?\n")
+            answer = self.s.readline().strip()
+            if answer and len(answer) > 1:
+                # Remove the first character (command echo) and return the value
+                return answer[1:]
+            if attempt < retries - 1:
+                time.sleep(retry_delay)
         return None
 
     def get_value(self, name):
@@ -134,9 +139,17 @@ class Lakeshore(TemperatureController):
 
     def set_setpoint(self, temperature: float) -> bool:
         """Set target temperature in Celsius."""
-        self.s.write(f"SETP {temperature}\n")
+        self.s.write(f"SETP 1,{temperature}\n")
         return True
 
+    def check_connection(self) -> bool:
+        """Check if the Lakeshore device is connected and responsive."""
+        try:
+            result = self.read_param('KRDG')
+            return result is not None
+        except Exception:
+            return False
+
     def close(self):
         """Close the serial connection."""
         if hasattr(self, 's') and self.s is not None:

src/damaris/tools/temperature.py

@@ -149,3 +149,13 @@ class TemperatureController(ABC):
         if they have resources to clean up.
         """
         pass
+
+    def check_connection(self) -> bool:
+        """
+        Check if the controller is connected and responsive.
+        
+        Returns:
+            bool: True if the controller is connected and responsive,
+                  False otherwise. Subclasses should override this method.
+        """
+        return False

tests/exp_test.py

@@ -11,5 +11,7 @@ def test(i):
     return e
 
 def experiment():
-    for i in range(10):
-        yield test(i)
+    for i in range(100):
+        #sleep(0.1)
+        yield test(i)
+        synchronize()

tests/profile_synchronize.py

@@ -0,0 +1,500 @@
+#!/usr/bin/env python3
+"""
+Profile the synchronize() round-trip: experiment script -> job file -> result file -> result script.
+
+Measures time spent in:
+  1. synchronize() polling loop (ExperimentHandling)
+  2. BlockingResultReader polling (waiting for result files)
+  3. XML parsing + base64 decode (ResultReader)
+  4. File I/O (write job / read result)
+  5. ResultHandling iteration overhead
+
+Usage:
+  python profile_synchronize.py [--jobs N] [--samples M] [--spool DIR]
+
+Defaults: 100 jobs, 1024 samples, spool=/tmp/damaris_profile
+"""
+
+import os
+import sys
+import time
+import shutil
+import tempfile
+import threading
+import random
+import argparse
+from collections import defaultdict
+
+# Add src to path
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "src"))
+
+from damaris.experiments.Experiment import Experiment
+from damaris.gui.ExperimentWriter import ExperimentWriterWithCleanup
+from damaris.gui.ResultReader import BlockingResultReader
+from damaris.gui.ExperimentHandling import ExperimentHandling
+from damaris.gui.ResultHandling import ResultHandling
+
+# ---------------------------------------------------------------------------
+# Shared data pool (thread-safe via dict — same as real code)
+# ---------------------------------------------------------------------------
+data = {}
+
+# ---------------------------------------------------------------------------
+# Profiling helpers
+# ---------------------------------------------------------------------------
+class ProfileTimer:
+    """Simple per-section timer that accumulates across multiple calls."""
+    def __init__(self):
+        self.sections = defaultdict(float)       # section -> total seconds
+        self.calls = defaultdict(int)             # section -> call count
+        self.max_time = defaultdict(float)        # section -> max single call
+        self._lock = threading.Lock()
+
+    def start(self, section):
+        self._section = section
+        self._start = time.perf_counter()
+
+    def stop(self):
+        elapsed = time.perf_counter() - self._start
+        with self._lock:
+            self.sections[self._section] += elapsed
+            self.calls[self._section] += 1
+            if elapsed > self.max_time[self._section]:
+                self.max_time[self._section] = elapsed
+
+    def report(self):
+        print("\n" + "=" * 72)
+        print("PROFILE RESULTS")
+        print("=" * 72)
+        total = sum(self.sections.values())
+        print(f"{'Section':<40} {'Total (ms)':>10} {'Calls':>8} {'Avg (ms)':>10} {'Max (ms)':>10}")
+        print("-" * 72)
+        for section in sorted(self.sections, key=lambda s: self.sections[s], reverse=True):
+            t = self.sections[section] * 1000
+            c = self.calls[section]
+            avg = t / c if c else 0
+            mx = self.max_time[section] * 1000
+            print(f"{section:<40} {t:>10.2f} {c:>8} {avg:>10.2f} {mx:>10.2f}")
+        print("-" * 72)
+        print(f"{'TOTAL':<40} {total*1000:>10.2f}")
+        print("=" * 72)
+
+
+profile = ProfileTimer()
+
+# ---------------------------------------------------------------------------
+# Experiment script (simulated — generates jobs without hardware)
+# ---------------------------------------------------------------------------
+def make_experiment_script(num_jobs, samples):
+    """Return a string of an experiment function that generates jobs."""
+    return f"""
+def experiment():
+    for i in range({num_jobs}):
+        e = Experiment()
+        e.ttl_pulse(length=1e-6, value=1)
+        e.wait(1e-3)
+        e.ttl_pulse(length=1e-6, value=1)
+        e.record(samples={samples}, frequency=1e6, sensitivity=1)
+        e.set_description("iteration", i)
+        yield e
+        synchronize()
+"""
+
+# ---------------------------------------------------------------------------
+# Result script (simulated — just counts results)
+# ---------------------------------------------------------------------------
+def make_result_script():
+    return """
+def result():
+    count = 0
+    for ts in results:
+        count += 1
+        data["result_count"] = count
+"""
+
+# ---------------------------------------------------------------------------
+# Simulated backend: writes result files after a short delay
+# ---------------------------------------------------------------------------
+def simulate_backend(spool_dir, writer, result_reader, timer):
+    """
+    Simulates the hardware backend: reads jobs from spool, processes them,
+    and writes result files. In real usage this is external, but for profiling
+    we simulate it to measure the full round-trip.
+    """
+    # We don't actually run the backend here — instead we profile the real
+    # synchronize() flow by running ExperimentHandling and ResultHandling
+    # against a mock result source.
+    pass
+
+
+# ---------------------------------------------------------------------------
+# Real profiling: run ExperimentHandling + ResultHandling with mock results
+# ---------------------------------------------------------------------------
+def profile_roundtrip(num_jobs=100, samples=1024, spool_dir=None):
+    """
+    Profile the synchronize() round-trip by running the real ExperimentHandling
+    and ResultHandling threads, with a mock result generator that simulates
+    the backend writing result files.
+    """
+    global data
+    data = {}
+    data["__recentexperiment"] = -1
+    data["__recentresult"] = -1
+
+    if spool_dir is None:
+        spool_dir = tempfile.mkdtemp(prefix="damaris_profile_")
+
+    print(f"Spool directory: {spool_dir}")
+    print(f"Jobs: {num_jobs}, Samples per job: {samples}")
+    print()
+
+    # --- Create writer and reader ---
+    writer = ExperimentWriterWithCleanup(spool_dir, inform_last_job=None)
+    reader = BlockingResultReader(spool_dir)
+    reader.poll_time = 0.05  # 50ms polling
+
+    # --- Experiment script ---
+    exp_script = make_experiment_script(num_jobs, samples)
+
+    # --- Result script ---
+    res_script = make_result_script()
+
+    # --- Start threads ---
+    exp_handler = ExperimentHandling(exp_script, writer, data)
+    res_handler = ResultHandling(res_script, reader, data)
+
+    # --- Mock backend: write result files in a separate thread ---
+    backend_done = threading.Event()
+
+    def mock_backend():
+        """Simulates hardware backend writing result files."""
+        try:
+            # Read each job file, create a result, write it
+            job_no = 0
+            while not backend_done.is_set() or True:
+                result_file = os.path.join(spool_dir, f"job.{job_no:09d}.result")
+                job_file = os.path.join(spool_dir, f"job.{job_no:09d}")
+
+                if not os.path.exists(job_file):
+                    if backend_done.is_set() and job_no >= writer.no:
+                        break
+                    time.sleep(0.01)
+                    continue
+
+                # Simulate some processing delay (like real hardware)
+                processing_delay = random.uniform(0.001, 0.010)  # 1-10ms
+                time.sleep(processing_delay)
+
+                # Write result file (simplified XML)
+                timer = ProfileTimer()
+                timer.start("result_file_io")
+                with open(result_file, "w") as f:
+                    f.write(f'<result job="{job_no}">\n')
+                    f.write(f'  <adcdata rate="1000000.0" channels="2" samples="{samples}">\n')
+                    # Generate fake base64-like data
+                    import base64
+                    fake_data = bytes([random.randint(-128, 127) for _ in range(samples * 2)])
+                    encoded = base64.b64encode(fake_data).decode("ascii")
+                    # Write in 62-char lines like real XML
+                    for i in range(0, len(encoded), 62):
+                        f.write(encoded[i:i+62] + "\n")
+                    f.write(f'  </adcdata>\n')
+                    f.write(f'</result>\n')
+                timer.stop()
+                # Merge result_file_io into profile
+                for sec, t in timer.sections.items():
+                    profile.sections[sec] += t
+                    profile.calls[sec] += 1
+                    if t > profile.max_time[sec]:
+                        profile.max_time[sec] = t
+
+                job_no += 1
+        except Exception as e:
+            print(f"Backend error: {e}")
+            import traceback
+            traceback.print_exc()
+
+    backend_thread = threading.Thread(target=mock_backend, name="mock_backend")
+    backend_thread.start()
+
+    # --- Patch synchronize to profile the polling loop ---
+    original_synchronize = exp_handler.synchronize
+
+    def profiled_synchronize(before=0, waitsteps=0.1):
+        profile.start("synchronize_polling")
+        iterations = 0
+        while (data["__recentexperiment"] > data["__recentresult"] + before) and not exp_handler.quit_flag.isSet():
+            iterations += 1
+            exp_handler.quit_flag.wait(waitsteps)
+        profile.stop("synchronize_polling")
+        profile.calls["synchronize_poll_iterations"] += iterations
+        if original_synchronize.__self__.quit_flag.isSet():
+            raise Exception("StopExperiment")
+
+    exp_handler.synchronize = profiled_synchronize
+
+    # --- Patch ResultReader to profile XML parsing ---
+    original_get_result = reader.get_result_object
+
+    def profiled_get_result(in_filename):
+        profile.start("result_file_read")
+        profile.start("xml_parsing")
+        result = original_get_result(in_filename)
+        profile.stop("xml_parsing")
+        profile.stop("result_file_read")
+        return result
+
+    reader.get_result_object = profiled_get_result
+
+    # --- Patch ResultHandling.__iter__ to profile iteration overhead ---
+    original_iter = res_handler.__iter__
+
+    def profiled_iter():
+        profile.start("result_iteration")
+        for item in original_iter():
+            profile.stop("result_iteration")
+            yield item
+            profile.start("result_iteration")
+
+    res_handler.__iter__ = profiled_iter
+
+    # --- Run ---
+    start_time = time.perf_counter()
+
+    exp_handler.start()
+    res_handler.start()
+
+    exp_handler.join(timeout=120)
+    res_handler.join(timeout=120)
+
+    backend_done.set()
+    backend_thread.join(timeout=10)
+
+    elapsed = time.perf_counter() - start_time
+
+    # --- Report ---
+    print(f"\nTotal wall time: {elapsed:.3f}s")
+    print(f"Jobs processed: {data.get('__recentexperiment', 0) + 1}")
+    print(f"Results processed: {data.get('__recentresult', 0) + 1}")
+
+    profile.report()
+
+    # --- Cleanup ---
+    shutil.rmtree(spool_dir, ignore_errors=True)
+
+    return profile
+
+
+# ---------------------------------------------------------------------------
+# Alternative: profile with real experiment/result scripts from tests/
+# ---------------------------------------------------------------------------
+def profile_with_real_scripts(spool_dir=None):
+    """
+    Profile using the real exp_test.py and res_test.py scripts.
+    This requires the actual backend to be running.
+    """
+    if spool_dir is None:
+        spool_dir = tempfile.mkdtemp(prefix="damaris_profile_real_")
+
+    print(f"Spool directory: {spool_dir}")
+
+    # Read real scripts
+    with open(os.path.join(os.path.dirname(__file__), "tests", "exp_test.py")) as f:
+        exp_script = f.read()
+    with open(os.path.join(os.path.dirname(__file__), "tests", "res_test.py")) as f:
+        res_script = f.read()
+
+    data = {}
+    data["__recentexperiment"] = -1
+    data["__recentresult"] = -1
+
+    writer = ExperimentWriterWithCleanup(spool_dir)
+    reader = BlockingResultReader(spool_dir, poll_time=0.05)
+
+    exp_handler = ExperimentHandling(exp_script, writer, data)
+    res_handler = ResultHandling(res_script, reader, data)
+
+    exp_handler.start()
+    res_handler.start()
+
+    exp_handler.join(timeout=120)
+    res_handler.join(timeout=120)
+
+    print(f"\nJobs: {data.get('__recentexperiment', 0) + 1}")
+    print(f"Results: {data.get('__recentresult', 0) + 1}")
+
+    shutil.rmtree(spool_dir, ignore_errors=True)
+
+
+# ---------------------------------------------------------------------------
+# Standalone: profile synchronize polling without threads
+# ---------------------------------------------------------------------------
+def profile_synchronize_polling(num_jobs=100, before=0):
+    """
+    Profile just the synchronize() polling loop in isolation.
+    Simulates the gap between __recentexperiment and __recentresult.
+    """
+    print("\n" + "=" * 72)
+    print("ISOLATED SYNCHRONIZE POLLING PROFILE")
+    print("=" * 72)
+
+    data = {"__recentexperiment": 0, "__recentresult": 0}
+    quit_flag = threading.Event()
+
+    # Simulate: experiment advances faster than result
+    # Experiment is at job N, result is at job N-before
+    # synchronize() must wait for result to catch up
+
+    total_wait_time = 0
+    num_sync_calls = 0
+    total_poll_iterations = 0
+
+    for job_id in range(num_jobs):
+        data["__recentexperiment"] = job_id
+
+        # Simulate result lagging behind by 'before' jobs
+        # In real code, result catches up asynchronously
+        # We simulate this by having result advance at a fixed rate
+
+        wait_start = time.perf_counter()
+        iterations = 0
+        while (data["__recentexperiment"] > data["__recentresult"] + before) and not quit_flag.isSet():
+            iterations += 1
+            quit_flag.wait(0.1)  # the waitsteps parameter
+
+            # Simulate result catching up (in real code, this happens in ResultHandling)
+            if data["__recentresult"] < job_id - before:
+                data["__recentresult"] = min(job_id - before, data["__recentresult"] + 1)
+
+        wait_elapsed = time.perf_counter() - wait_start
+        total_wait_time += wait_elapsed
+        num_sync_calls += 1
+        total_poll_iterations += iterations
+
+    avg_wait = total_wait_time / num_sync_calls if num_sync_calls else 0
+    avg_iterations = total_poll_iterations / num_sync_calls if num_sync_calls else 0
+
+    print(f"Jobs: {num_jobs}, before: {before}")
+    print(f"synchronize() calls: {num_sync_calls}")
+    print(f"Total wait time: {total_wait_time*1000:.1f}ms")
+    print(f"Avg wait per call: {avg_wait*1000:.1f}ms")
+    print(f"Total poll iterations: {total_poll_iterations}")
+    print(f"Avg iterations per call: {avg_iterations:.1f}")
+    print(f"Poll overhead: ~{total_poll_iterations * 0.1 * 1000:.1f}ms of wake-up latency")
+    print()
+
+
+# ---------------------------------------------------------------------------
+# Profile XML parsing on realistic data
+# ---------------------------------------------------------------------------
+def profile_xml_parsing(num_jobs=100, samples=1024, channels=2, spool_dir=None):
+    """Profile just the XML parsing + base64 decode cost."""
+    print("\n" + "=" * 72)
+    print("XML PARSING PROFILE")
+    print("=" * 72)
+
+    if spool_dir is None:
+        spool_dir = tempfile.mkdtemp(prefix="damaris_xml_profile_")
+
+    import base64
+    import numpy
+
+    # Generate result files
+    for job_id in range(num_jobs):
+        result_file = os.path.join(spool_dir, f"job.{job_id:09d}.result")
+        with open(result_file, "w") as f:
+            f.write(f'<result job="{job_id}">\n')
+            f.write(f'  <adcdata rate="1000000.0" channels="{channels}" samples="{samples}">\n')
+            fake_data = bytes([random.randint(0, 255) for _ in range(samples * channels)])
+            encoded = base64.b64encode(fake_data).decode("ascii")
+            for i in range(0, len(encoded), 62):
+                f.write(encoded[i:i+62] + "\n")
+            f.write(f'  </adcdata>\n')
+            f.write(f'</result>\n')
+
+    # Profile parsing
+    reader = BlockingResultReader(spool_dir)
+    parse_times = []
+    decode_times = []
+    split_times = []
+
+    for job_id in range(num_jobs):
+        result_file = os.path.join(spool_dir, f"job.{job_id:09d}.result")
+
+        t0 = time.perf_counter()
+        result = reader.get_result_object(result_file)
+        total = time.perf_counter() - t0
+
+        # The parsing happens inside get_result_object
+        # We can't easily separate XML parse from decode in the current code
+        # but we can measure the total
+        parse_times.append(total)
+
+    avg_parse = numpy.mean(parse_times) * 1000
+    max_parse = numpy.max(parse_times) * 1000
+    total_parse = numpy.sum(parse_times) * 1000
+
+    print(f"Jobs: {num_jobs}, Samples: {samples}, Channels: {channels}")
+    print(f"Total parse time: {total_parse:.1f}ms")
+    print(f"Avg per result: {avg_parse:.2f}ms")
+    print(f"Max per result: {max_parse:.2f}ms")
+    print(f"Throughput: {num_jobs / (total_parse/1000):.0f} results/sec")
+    print()
+
+    shutil.rmtree(spool_dir, ignore_errors=True)
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Profile synchronize() round-trip")
+    parser.add_argument("--jobs", type=int, default=100, help="Number of jobs (default: 100)")
+    parser.add_argument("--samples", type=int, default=1024, help="Samples per job (default: 1024)")
+    parser.add_argument("--channels", type=int, default=2, help="Channels (default: 2)")
+    parser.add_argument("--spool", type=str, default=None, help="Spool directory")
+    parser.add_argument("--mode", choices=["full", "polling", "xml", "all"], default="all",
+                        help="Profiling mode (default: all)")
+    args = parser.parse_args()
+
+    if args.mode in ("polling", "all"):
+        # Profile synchronize polling with different lag values
+        for before in [0, 5, 10, 20]:
+            profile_synchronize_polling(num_jobs=args.jobs, before=before)
+
+    if args.mode in ("xml", "all"):
+        profile_xml_parsing(num_jobs=args.jobs, samples=args.samples, channels=args.channels, spool_dir=args.spool)
+
+    if args.mode in ("full",):
+        print("Full threaded profile requires a running backend. Skipping.")
+
+    if args.mode == "all":
+        print("\n" + "=" * 72)
+        print("SUMMARY OF FINDINGS")
+        print("=" * 72)
+        print("""
+The synchronize() round-trip bottleneck analysis:
+
+1. POLLING LATENCY (synchronize() + BlockingResultReader)
+   - synchronize() polls every 100ms (waitsteps=0.1)
+   - BlockingResultReader polls every 100ms (poll_time=0.1)
+   - Worst-case added latency: ~200ms per job (one full cycle of each poll)
+   - This is the PRIMARY bottleneck for small job counts
+
+2. XML PARSING + BASE64 DECODE
+   - Per-result overhead depends on sample count
+   - For 1024 samples, 2 channels: ~X ms per result
+   - Scales linearly with sample count
+
+3. FILE I/O
+   - Writing job files: minimal (atomic rename)
+   - Reading result files: depends on disk speed and result size
+
+4. RESULT HANDLING ITERATION
+   - Minimal overhead (dict updates + yield)
+
+RECOMMENDATION:
+- Reduce synchronize() waitsteps from 0.1 to 0.01 for lower latency
+- Reduce BlockingResultReader poll_time from 0.1 to 0.01
+- Consider using inotify/fsevents for file-system notifications instead of polling
+""")

tests/res_test.py

@@ -1,6 +1,6 @@
 import tables
 import numpy
-
+# from damaris import FitResult
 r = MeasurementResult("test")
 
 def result():
@@ -9,14 +9,29 @@ def result():
     except:
         print("h5 already opened")
     accu = Accumulation(error=False)
+
     for num,ts in enumerate(results):
         data["accu"] = accu
         data["data/ts %i"%num] = ts+0
         ts.write_to_hdf(h5, "/", f"adc_{num}", f"adc_{num}")
         data["fft"] = (ts+0).fft()
         r[num] = ts.y[0][0]
+        if num > 4:
+            x = r.get_xdata()
+            y = r.get_ydata()
+            coeffs = numpy.polyfit(x,  y, 2)
+            x_line = numpy.linspace(x.min(), x.max(), 100)
+            y_line = numpy.polyval(coeffs, x_line)
+            
+            fit = FitResult(x=x_line,
+                    y=y_line,
+                    title="Magnetization(tau) fit",
+                    source="test",
+                    params={"a": coeffs[0], "b": coeffs[1], "c": coeffs[2]})
+            data["test/fit"] = fit
         data["test"] = r
         accu+=ts
+        print(accu)
     accu.write_to_hdf(h5, "/", "name", "title")
     try:
         h5.close()

tests/test_ADC_Result.py

@@ -64,6 +64,7 @@ class TestADCResult(unittest.TestCase):
         self.assertTrue(np.array_equal(adc.x, np.zeros(3, dtype="float32")))
         self.assertTrue(np.array_equal(adc.y[0], np.zeros(3, dtype="int16")))
         self.assertEqual(adc.index, [(0, 2)])
+        self.assertEqual(adc.nChannels, 2)
 
     def test_add_sample_space(self):
         """Test adding sample space."""
@@ -90,6 +91,33 @@ class TestADCResult(unittest.TestCase):
         self.assertEqual(sub_result.description, {"key": "value"})
         self.assertEqual(sub_result.job_id, 42)
 
+    def test_get_result_by_index_copies_nChannels_and_is_clipped(self):
+        """Test that get_result_by_index properly copies nChannels and is_clipped."""
+        adc = self.create_adc_result()
+        adc.index = [(0, 1), (1, 2)]
+        adc.set_nChannels(2)
+        adc.is_clipped = True
+
+        sub_result = adc.get_result_by_index(1)
+
+        self.assertEqual(sub_result.nChannels, 2)
+        self.assertEqual(sub_result.is_clipped, True)
+
+    def test_get_result_by_index_data_independence(self):
+        """Test that get_result_by_index returns independent data (not shared views)."""
+        adc = self.create_adc_result()
+        adc.index = [(0, 1), (1, 2)]
+        original_x = adc.x.copy()
+        original_data = adc.y[0].copy()
+
+        sub_result = adc.get_result_by_index(1)
+        sub_result.y[0][0] = 9999
+        sub_result.x[0] = 999.0
+
+        self.assertEqual(sub_result.y[0][0], 9999)
+        self.assertTrue(np.array_equal(adc.y[0], original_data))
+        self.assertTrue(np.array_equal(adc.x, original_x))
+
     def test_set_sampling_rate(self):
         """Test updating the sampling rate."""
         adc = ADC_Result()
@@ -167,14 +195,14 @@ class TestADCResult(unittest.TestCase):
             np.testing.assert_array_equal(adc.y[i]+10, y[i] + 10, strict=True)
         adc += 10
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] + 10, strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] + 10).astype("float32"), strict=True)
 
         # test float addition
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i] + 10., y[i] + 10., strict=True)
+            np.testing.assert_array_equal(adc.y[i] + 10., (y[i] + 10 + 10.).astype("float32"), strict=True)
         adc += 10.
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] + 10 + 10., strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] + 10 + 10.).astype("float32"), strict=True)
 
     def test_operator_sub_scalar(self):
         """
@@ -187,14 +215,14 @@ class TestADCResult(unittest.TestCase):
             np.testing.assert_array_equal(adc.y[i] - 10, y[i] - 10, strict=True)
         adc -= 10
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] - 10, strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] - 10).astype("float32"), strict=True)
 
         # test float subtraction
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i] - 10., y[i] - 10., strict=True)
+            np.testing.assert_array_equal(adc.y[i] - 10., (y[i] - 10 - 10.).astype("float32"), strict=True)
         adc -= 10.
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] - 10 - 10., strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] - 10 - 10.).astype("float32"), strict=True)
 
     def test_operator_mul_scalar(self):
         """
@@ -208,14 +236,14 @@ class TestADCResult(unittest.TestCase):
             np.testing.assert_array_equal(adc.y[i] * 10, y[i] * 10, strict=True)
         adc *= 10
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] * 10, strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] * 10).astype("float32"), strict=True)
 
         # test float multiplication
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i] * 10.0, y[i] * 10.0, strict=True)
+            np.testing.assert_array_equal(adc.y[i] * 10.0, (y[i] * 10 * 10.0).astype("float32"), strict=True)
         adc *= 10.0
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] * 10 * 10.0, strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] * 10 * 10.0).astype("float32"), strict=True)
 
     def test_operator_truediv_scalar(self):
         """
@@ -227,7 +255,7 @@ class TestADCResult(unittest.TestCase):
             np.testing.assert_array_equal(adc.y[i]/10, y[i] / 10, strict=True)
         adc /= 10
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] / 10, strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] / 10).astype("float32"), strict=True)
 
     def test_operator_floordiv_scalar(self):
         """
@@ -239,10 +267,37 @@ class TestADCResult(unittest.TestCase):
             np.testing.assert_array_equal(adc.y[i]//10, y[i] // 10, strict=True)
         adc //= 10
         for i in range(adc.get_nChannels()):
-            np.testing.assert_array_equal(adc.y[i], y[i] // 10, strict=True)
+            np.testing.assert_array_equal(adc.y[i], (y[i] // 10).astype("float32"), strict=True)
         self.assertRaises(ValueError, adc.__floordiv__, 1.0)
         self.assertRaises(ValueError, adc.__rfloordiv__, 1.0)
 
+    def test_operators_keep_channel_metadata(self):
+        """operator results keep nChannels/is_clipped identical to the operand"""
+        adc = self.create_adc_result()
+        adc.is_clipped = True
+        results = [adc + 1, adc - 1, adc * 2, adc ** 2, adc / 2., adc // 2,
+                   10 + adc, 10 - adc, 2 * adc, 100. / adc, 100 // adc, -adc]
+        for r in results:
+            self.assertEqual(r.get_nChannels(), adc.get_nChannels())
+            self.assertEqual(r.is_clipped, adc.is_clipped)
+
+    def test_hdf_roundtrip_keeps_nchannels(self):
+        """read_from_hdf must restore the number of channels"""
+        import tempfile
+        import tables
+        from damaris.data.ADC_Result import read_from_hdf
+
+        adc = self.create_adc_result()
+        with tempfile.TemporaryDirectory() as tmpdir:
+            path = os.path.join(tmpdir, "adc.h5")
+            with tables.open_file(path, "w") as h5:
+                adc.write_to_hdf(h5, h5.root, "adc", "test")
+            with tables.open_file(path, "r") as h5:
+                adc2 = read_from_hdf(h5.root.adc)
+        self.assertIsNotNone(adc2)
+        self.assertTrue(adc2.contains_data())
+        self.assertEqual(adc2.get_nChannels(), adc.get_nChannels())
+
     def test_operator_overload_adc_result_exception(self):
         """
         Test the ValueError raised when ADC_Result is used as an operand in an arithmetic operation
@@ -261,5 +316,31 @@ class TestADCResult(unittest.TestCase):
         self.assertRaises(ValueError, adc1.__rfloordiv__, adc2)
         self.assertRaises(ValueError, adc1.__pow__, adc2)
 
+    def test_empty_state_types(self):
+        """Empty ADC_Result must hold an ndarray x and a list-of-ndarray y."""
+        adc = ADC_Result()
+        self.assertIsInstance(adc.x, np.ndarray)
+        self.assertIsInstance(adc.y, list)
+        self.assertEqual(adc.x.size, 0)
+        self.assertEqual(adc.get_xmin(), 0)
+        self.assertEqual(adc.get_xmax(), 0)
+        self.assertEqual(adc.get_xminpos(), 0)
+        self.assertEqual(adc.get_ymin(), 0)
+        self.assertEqual(adc.get_ymax(), 0)
+        self.assertEqual(adc.get_yminpos(), 0)
+
+    def test_constructor_coerces_lists(self):
+        """Plain lists passed to the constructor must become numpy arrays."""
+        x = [0.0, 1.0, 2.0]
+        y = [[10, 20, 30], [15, 25, 35]]
+        index = [(0, 2)]
+        adc = ADC_Result(x, y, index, 1000.0, {"key": "value"}, 1, datetime(2023, 1, 1))
+        self.assertIsInstance(adc.x, np.ndarray)
+        np.testing.assert_array_equal(adc.x, [0.0, 1.0, 2.0])
+        for ch in adc.y:
+            self.assertIsInstance(ch, np.ndarray)
+        np.testing.assert_array_equal(adc.y[0], [10, 20, 30])
+        np.testing.assert_array_equal(adc.y[1], [15, 25, 35])
+
 if __name__ == "__main__":
     unittest.main()

tests/test_Accumulation.py

@@ -1,5 +1,6 @@
 import os
 import subprocess
+import threading
 import unittest
 from datetime import datetime
 
@@ -107,6 +108,35 @@ class TestAccumulation(unittest.TestCase):
         self.assertEqual(sub_result.index, [(0, 1)])
         self.assertEqual(sub_result.common_descriptions, {"key": "value"})
 
+    def test_get_accu_by_index_copies_is_clipped(self):
+        """Test that get_accu_by_index properly copies is_clipped."""
+        adc = self.create_adc_result()
+        adc.index = [(0, 1), (1, 2)]
+        accu = Accumulation()
+        accu += adc
+        accu.is_clipped = True
+
+        sub_result = accu.get_accu_by_index(1)
+
+        self.assertEqual(sub_result.is_clipped, True)
+
+    def test_get_accu_by_index_data_independence(self):
+        """Test that get_accu_by_index returns independent data (not shared views)."""
+        adc = self.create_adc_result()
+        adc.index = [(0, 1), (1, 2)]
+        accu = Accumulation()
+        accu += adc
+        original_y0 = accu.y[0].copy()
+
+        sub_result = accu.get_accu_by_index(1)
+        sub_result.y[0][0] = 9999
+        sub_result.x[0] = 999.0
+
+        self.assertEqual(sub_result.y[0][0], 9999)
+        self.assertTrue(np.array_equal(accu.y[0], original_y0))
+        # x array should also be independent
+        self.assertEqual(accu.x[1], 1.0)
+
     def test_write_to_csv_without_error(self):
         """Test the functionality of writing to CSV."""
         from io import StringIO
@@ -229,7 +259,7 @@ class TestAccumulation(unittest.TestCase):
             np.testing.assert_array_equal(accu.y[i], y[i] - 10)
 
         # test float subtraction
-        adc -= 10.
+        accu -= 10.
         for i in range(adc.get_nChannels()):
             np.testing.assert_array_equal(accu.y[i], y[i] - 10 - 10.)
 
@@ -283,5 +313,252 @@ class TestAccumulation(unittest.TestCase):
         self.assertRaises(ValueError, accu.__floordiv__, 1.0)
         self.assertRaises(ValueError, accu.__ifloordiv__, 1.0)
 
+    def test_add_invalid_operand_raises(self):
+        """accu + bogus must raise ValueError instead of returning None."""
+        adc = self.create_adc_result()
+        accu = Accumulation()
+        accu += adc
+        self.assertRaises(ValueError, accu.__add__, "foo")
+        self.assertRaises(ValueError, accu.__iadd__, "foo")
+
+    def test_neg_empty_raises(self):
+        """-empty accumulation must raise ValueError."""
+        self.assertRaises(ValueError, Accumulation().__neg__)
+
+    def test_add_adc_result_does_not_modify_job_ids(self):
+        """accu + adc must not insert the adc job id into accu.job_ids."""
+        accu = Accumulation()
+        accu += self.create_adc_result()
+        n_before = len(accu.job_ids)
+        adc2 = self.create_adc_result()
+        adc2.job_id = 12
+        r = accu + adc2
+        self.assertIn(12, r.job_ids)
+        self.assertNotIn(12, accu.job_ids)
+        self.assertEqual(len(accu.job_ids), n_before)
+
+    def test_empty_plus_accumulation_copies_data(self):
+        """empty_accu + accu must copy arrays and metadata, not alias them."""
+        adc = self.create_adc_result()
+        accu = Accumulation()
+        accu += adc
+        accu.is_clipped = True
+
+        r = Accumulation() + accu
+        self.assertIsNot(r.y[0], accu.y[0])
+        self.assertIsNot(r.job_ids, accu.job_ids)
+        self.assertTrue(r.is_clipped)
+        # in-place operation on the result must leave the operand unchanged
+        r *= 2
+        np.testing.assert_array_equal(accu.y[0], np.array([10, 20, 30], dtype="float32"))
+
+    def test_accumulation_plus_accumulation_keeps_both_job_ids(self):
+        """accu1 + accu2 must keep the provenance of both operands."""
+        adc1 = self.create_adc_result()
+        adc2 = self.create_adc_result()
+        adc2.job_id = 12
+        accu1 = Accumulation()
+        accu1 += adc1
+        accu2 = Accumulation()
+        accu2 += adc2
+        r = accu1 + accu2
+        self.assertEqual(set(r.job_ids.keys()), {11, 12})
+        self.assertEqual(set(accu1.job_ids.keys()), {11})
+        self.assertEqual(set(accu2.job_ids.keys()), {12})
+
+    def test_empty_error_plus_non_error_raises(self):
+        """error accumulation + non-error accumulation must raise ValueError."""
+        accu = Accumulation()
+        accu += self.create_adc_result()
+        empty_err = Accumulation(error=True)
+        self.assertRaises(ValueError, empty_err.__add__, accu)
+        free = []
+        def check_lock():
+            if empty_err.lock.acquire(timeout=1):
+                free.append(True)
+                empty_err.lock.release()
+        t = threading.Thread(target=check_lock)
+        t.start()
+        t.join(3)
+        self.assertTrue(free, "lock leaked after ValueError in __add__")
+
+    def test_neg_keeps_y_square_as_list(self):
+        """-accu must store y_square as a list of per-channel arrays."""
+        x = np.array([0.0, 1.0, 2.0])
+        y = [np.array([10, 20, 30], dtype="float32")]
+        y_square = [np.array([100.0, 400.0, 900.0], dtype="float32")]
+        accu = Accumulation(x=x, y=y, y_2=y_square, n=2, index=[(0, 2)], sampl_freq=1000.0, error=True)
+
+        r = -accu
+
+        self.assertIsInstance(r.y_square, list)
+        # sums of squares are invariant under negation
+        np.testing.assert_array_equal(r.get_ysquare(0), y_square[0])
+
+    def test_iadd_adc_after_ndarray_y_2(self):
+        """empty accu with ndarray y_square += adc must not crash (was AttributeError)."""
+        y_square_2d = np.array([[100.0, 400.0, 900.0]], dtype="float32")
+        accu = Accumulation(y_2=y_square_2d, error=True)
+
+        accu += self.create_adc_result()
+
+        self.assertIsInstance(accu.y_square, list)
+        np.testing.assert_array_equal(accu.get_ydata(0), np.array([10, 20, 30], dtype="float32"))
+        np.testing.assert_array_equal(accu.get_ysquare(0), np.array([10, 20, 30], dtype="float32") ** 2)
+
+    def test_iadd_accumulation_after_ndarray_y_2(self):
+        """empty accu with ndarray y_square += accu must not crash (was AttributeError)."""
+        full = Accumulation(error=True)
+        full += self.create_adc_result()
+        y_square_2d = np.array([[100.0, 400.0, 900.0]], dtype="float32")
+        accu = Accumulation(y_2=y_square_2d, error=True)
+
+        accu += full
+
+        self.assertIsInstance(accu.y_square, list)
+        np.testing.assert_array_equal(accu.get_ydata(0), np.array([10, 20, 30], dtype="float32"))
+        np.testing.assert_array_equal(accu.get_ysquare(0), np.array([10, 20, 30], dtype="float32") ** 2)
+
+    def test_constructor_normalizes_ndarray_y_2(self):
+        """an ndarray passed as y_2 must be stored as a list of channel arrays."""
+        x = np.array([0.0, 1.0, 2.0])
+        y = [np.array([10, 20, 30], dtype="float32")]
+        y_square_2d = np.array([[100.0, 400.0, 900.0]], dtype="float32")
+
+        accu = Accumulation(x=x, y=y, y_2=y_square_2d, n=2, index=[(0, 2)], sampl_freq=1000.0, error=True)
+
+        self.assertIsInstance(accu.y_square, list)
+        np.testing.assert_array_equal(accu.get_ysquare(0), y_square_2d[0])
+
+    def test_add_accumulations_without_time_period(self):
+        """accu + accu without time information must not crash."""
+        accu1 = Accumulation()
+        accu1 += self.create_adc_result()
+        accu2 = Accumulation()
+        accu2 += self.create_adc_result()
+        accu1.time_period = None
+        accu2.time_period = []
+
+        r = accu1 + accu2
+
+        self.assertIsNone(r.time_period)
+
+    def test_iadd_accumulation_without_time_period(self):
+        """accu += accu without time information must not crash (was IndexError)."""
+        accu1 = Accumulation()
+        accu1 += self.create_adc_result()
+        accu2 = Accumulation()
+        accu2 += self.create_adc_result()
+        accu1.time_period = []
+        accu2.time_period = []
+
+        accu1 += accu2
+
+        self.assertIsNone(accu1.time_period)
+
+    def test_add_adc_without_time_period(self):
+        """accu + adc without time information must not crash."""
+        accu = Accumulation()
+        accu += self.create_adc_result()
+        accu.time_period = []
+
+        r = accu + self.create_adc_result()
+
+        self.assertIsNone(r.time_period)
+
+    def test_time_period_merged_on_add(self):
+        """accu + accu with both time periods set merges to min/max."""
+        accu1 = Accumulation()
+        accu1 += self.create_adc_result()
+        accu2 = Accumulation()
+        accu2 += self.create_adc_result()
+        accu1.time_period = [datetime(2023, 1, 3), datetime(2023, 1, 5)]
+        accu2.time_period = [datetime(2023, 1, 2), datetime(2023, 1, 7)]
+
+        r = accu1 + accu2
+
+        self.assertEqual(r.time_period, [datetime(2023, 1, 2), datetime(2023, 1, 7)])
+
+    def test_time_period_merged_with_adc_job_date(self):
+        """accu + adc merges self.time_period with other.job_date."""
+        accu = Accumulation()
+        accu += self.create_adc_result()
+        accu.time_period = [datetime(2023, 1, 3), datetime(2023, 1, 5)]
+        adc = self.create_adc_result()
+        adc.job_date = datetime(2023, 1, 1)
+
+        r = accu + adc
+
+        self.assertEqual(r.time_period, [datetime(2023, 1, 1), datetime(2023, 1, 5)])
+
+    def test_iadd_adc_without_job_date_sets_time_period_none(self):
+        """empty accu += adc without job_date gets time_period None."""
+        accu = Accumulation()
+        adc = self.create_adc_result()
+        adc.job_date = None
+
+        accu += adc
+
+        self.assertIsNone(accu.time_period)
+
+    def test_get_accu_by_index_without_time_period(self):
+        """get_accu_by_index without time information must not crash."""
+        accu = Accumulation()
+        adc = self.create_adc_result()
+        adc.index = [(0, 1), (1, 2)]
+        accu += adc
+        accu.time_period = None
+
+        r = accu.get_accu_by_index(0)
+
+        self.assertIsNone(r.time_period)
+
+    def test_write_to_tecmag_without_time_period_raises(self):
+        """write_to_tecmag without time information must raise ValueError."""
+        accu = Accumulation()
+        accu += self.create_adc_result()
+        accu.time_period = None
+
+        self.assertRaises(ValueError, accu.write_to_tecmag, "/tmp/damaris_tecmag_no_time.tnt")
+        self.assertTrue(accu.lock.acquire(timeout=1), "lock leaked after ValueError in write_to_tecmag")
+        accu.lock.release()
+
+    def test_empty_state_accessors(self):
+        """Empty Accumulation accessors must return ndarrays, min/max helpers 0."""
+        accu = Accumulation()
+        self.assertIsInstance(accu.x, np.ndarray)
+        self.assertEqual(accu.x.size, 0)
+        self.assertIsInstance(accu.get_ydata(), np.ndarray)
+        self.assertEqual(accu.get_ydata().size, 0)
+        self.assertIsInstance(accu.get_yerr(0), np.ndarray)
+        self.assertEqual(accu.get_yerr(0).size, 0)
+        self.assertEqual(accu.get_xmin(), 0)
+        self.assertEqual(accu.get_xmax(), 0)
+        self.assertEqual(accu.get_ymin(), 0)
+        self.assertEqual(accu.get_ymax(), 0)
+
+    def test_get_yerr_without_statistics_on_empty(self):
+        """get_yerr on an empty accumulation without statistics must not crash."""
+        accu = Accumulation(error=False)
+        err = accu.get_yerr(0)
+        self.assertIsInstance(err, np.ndarray)
+        self.assertEqual(err.size, 0)
+
+    def test_constructor_coerces_lists(self):
+        """Plain lists passed to the constructor must become numpy arrays."""
+        x = [0.0, 1.0, 2.0]
+        y = [[10, 20, 30], [15, 25, 35]]
+        y_2 = [[100.0, 400.0, 900.0], [225.0, 625.0, 1225.0]]
+        accu = Accumulation(x=x, y=y, y_2=y_2, n=1,
+                            index=[(0, 2)], sampl_freq=1000.0, error=True)
+        self.assertIsInstance(accu.x, np.ndarray)
+        np.testing.assert_array_equal(accu.x, [0.0, 1.0, 2.0])
+        for ch in accu.y:
+            self.assertIsInstance(ch, np.ndarray)
+        for ch in accu.y_square:
+            self.assertIsInstance(ch, np.ndarray)
+        np.testing.assert_array_equal(accu.y[0], [10, 20, 30])
+        np.testing.assert_array_equal(accu.y_square[0], [100.0, 400.0, 900.0])
+
 if __name__ == "__main__":
     unittest.main()

tests/test_Config_Result.py

@@ -0,0 +1,29 @@
+import unittest
+
+from damaris.data.Config_Result import Config_Result
+
+
+class TestConfigResult(unittest.TestCase):
+    def test_constructor_with_no_arguments(self):
+        """Test initializer with no arguments."""
+        cr = Config_Result()
+        self.assertEqual(cr.get_config_dictionary(), {})
+        self.assertEqual(cr.description, {})
+
+    def test_constructor_with_arguments(self):
+        """Test initializer with config and description arguments."""
+        config = {"a": 1}
+        desc = {"b": 2}
+        cr = Config_Result(config=config, desc=desc)
+        self.assertEqual(cr.get_config_dictionary(), config)
+        self.assertEqual(cr.description, desc)
+
+    def test_get_set_config(self):
+        cr = Config_Result()
+        cr.set_config("key", "value")
+        self.assertEqual(cr.get_config("key"), "value")
+        self.assertIsNone(cr.get_config("missing"))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_DamarisFFT.py

@@ -0,0 +1,44 @@
+import unittest
+
+import numpy as np
+
+from damaris.data.DamarisFFT import DamarisFFT
+
+
+class TestDamarisFFT(unittest.TestCase):
+    @staticmethod
+    def make_fft():
+        fft = DamarisFFT()
+        fft.sampling_rate = 100.0
+        fft.x = np.arange(10, dtype="float32") / 100.0
+        fft.y = [np.arange(10, dtype="float32")]
+        fft.xlabel = "Time / s"
+        return fft
+
+    def test_clip_without_arguments_is_placeholder(self):
+        """clip() without arguments returns self unchanged (placeholder use)."""
+        fft = self.make_fft()
+        self.assertIs(fft.clip(), fft)
+        self.assertEqual(len(fft.x), 10)
+        np.testing.assert_array_equal(fft.y[0], np.arange(10, dtype="float32"))
+
+    def test_clip_swaps_reversed_bounds(self):
+        fft = self.make_fft()
+        fft.clip(start=0.05, stop=0.01)
+        expected_x = (np.arange(10, dtype="float32") / 100.0)[1:5]
+        np.testing.assert_array_equal(fft.x, expected_x)
+        np.testing.assert_array_equal(fft.y[0], np.arange(1, 5, dtype="float32"))
+
+    def test_clip_single_bound(self):
+        fft = self.make_fft()
+        fft.clip(start=0.02)
+        self.assertEqual(len(fft.x), 7)
+        self.assertEqual(len(fft.y[0]), 7)
+
+    def test_clip_too_close_raises(self):
+        fft = self.make_fft()
+        self.assertRaises(ValueError, fft.clip, 0.021, 0.029)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_Drawable.py

@@ -0,0 +1,54 @@
+import unittest
+
+import numpy as np
+
+from damaris.data.Drawable import Drawable
+
+
+class TestDrawable(unittest.TestCase):
+    def test_initial_types(self):
+        """x must start as an empty ndarray, y as a list of arrays."""
+        d = Drawable()
+        self.assertIsInstance(d.x, np.ndarray)
+        self.assertEqual(d.x.size, 0)
+        self.assertIsInstance(d.y, list)
+
+    def test_empty_min_max_helpers(self):
+        """min/max helpers must return 0 instead of raising on empty data."""
+        d = Drawable()
+        self.assertEqual(d.get_xmin(), 0)
+        self.assertEqual(d.get_xmax(), 0)
+        self.assertEqual(d.get_xminpos(), 0)
+        self.assertEqual(d.get_ymin(), 0)
+        self.assertEqual(d.get_ymax(), 0)
+        self.assertEqual(d.get_yminpos(), 0)
+
+    def test_min_max_helpers_with_data(self):
+        d = Drawable()
+        d.x = np.array([-1.0, 0.0, 2.0, 5.0])
+        d.y = [np.array([1.0, -2.0, 3.0]), np.array([0.5, 4.0, -0.5])]
+        self.assertEqual(d.get_xmin(), -1.0)
+        self.assertEqual(d.get_xmax(), 5.0)
+        self.assertEqual(d.get_xminpos(), 2.0)
+        self.assertEqual(d.get_ymin(), -2.0)
+        self.assertEqual(d.get_ymax(), 4.0)
+        self.assertEqual(d.get_yminpos(), 0.5)
+
+    def test_yminpos_without_positive_values(self):
+        d = Drawable()
+        d.x = np.array([1.0])
+        d.y = [np.array([-1.0, -2.0])]
+        self.assertEqual(d.get_yminpos(), 0)
+
+    def test_set_xdata_set_ydata(self):
+        d = Drawable()
+        d.x = np.zeros(3)
+        d.y = [np.zeros(3)]
+        d.set_xdata(1, 5.0)
+        d.set_ydata(0, 1, 7.0)
+        self.assertEqual(d.get_xdata()[1], 5.0)
+        self.assertEqual(d.get_ydata(0)[1], 7.0)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_Error_Result.py

@@ -0,0 +1,23 @@
+import unittest
+
+import numpy as np
+
+from damaris.data.Error_Result import Error_Result
+
+
+class TestErrorResult(unittest.TestCase):
+    def test_plotdata_are_ndarrays(self):
+        """get_xdata/get_ydata must return numpy arrays like every Drawable."""
+        er = Error_Result("core failed")
+        self.assertIsInstance(er.get_xdata(), np.ndarray)
+        self.assertIsInstance(er.get_ydata(), np.ndarray)
+        np.testing.assert_array_equal(er.get_xdata(), [0.0])
+        np.testing.assert_array_equal(er.get_ydata(), [0.0])
+
+    def test_add_returns_self(self):
+        er = Error_Result("msg")
+        self.assertIs(er + 1, er)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_Experiment.py

@@ -0,0 +1,75 @@
+import re
+import unittest
+
+from damaris.experiments.Experiment import Experiment
+
+
+def phase_of_state(state):
+    "extracts the phase value from a state's XML content"
+    match = re.search(r'phase="([-0-9.]+)"', state.content)
+    assert match, "no phase in state: %s" % state.content
+    return float(match.group(1))
+
+
+class TestExperimentPhase(unittest.TestCase):
+    def test_set_phase_negative_normalized(self):
+        """Test that set_phase() normalizes negative phases to [0, 360)."""
+        e = Experiment()
+        for degrees, expected in ((-90, 270.0), (-45.5, 314.5),
+                                  (-270, 90.0), (-360, 0.0)):
+            e.set_phase(degrees)
+            self.assertEqual(phase_of_state(e.state_list.list[-1]), expected)
+
+    def test_set_phase_positive_unchanged(self):
+        """Test that set_phase() passes positive phases through unchanged."""
+        e = Experiment()
+        for degrees, expected in ((0, 0.0), (90, 90.0), (270, 270.0),
+                                  (45.5, 45.5)):
+            e.set_phase(degrees)
+            self.assertEqual(phase_of_state(e.state_list.list[-1]), expected)
+
+    def test_set_frequency_negative_phase_normalized(self):
+        """Test that set_frequency() normalizes a negative phase."""
+        e = Experiment()
+        e.set_frequency(85e6, -90)
+        state = e.state_list.list[-1]
+        self.assertEqual(phase_of_state(state), 270.0)
+        self.assertIn('f="85000000.000000"', state.content)
+
+    def test_rf_pulse_negative_phase_normalized(self):
+        """Test that rf_pulse() normalizes a negative pulse phase."""
+        e = Experiment(gating=1e-6, rf_sources=[1], rf_gates=[2])
+        e.rf_pulse(length=1e-6, phase=-90, source=0)
+        self.assertEqual(phase_of_state(e.state_list.list[0]), 270.0)
+
+    def test_numpy_scalars_in_description(self):
+        """Test that description items with numpy scalars (e.g. from lin_range)
+        are written as plain numbers so the result script can parse them again."""
+        import numpy as np
+        from damaris.tools.ranges import lin_range
+        e = Experiment()
+        e.set_description("tau", lin_range(1e-6, 3e-6, 1e-6)[0])
+        e.set_description("n", np.int64(42))
+        xml = e.write_xml_string()
+        items = {key: (typ, text)
+                 for key, typ, text in
+                 re.findall(r'<item key="([^"]+)" type="([^"]+)">([^<]*)</item>', xml)}
+        self.assertEqual(items["tau"][0], "Float")
+        self.assertEqual(items["n"][0], "Int")
+        self.assertEqual(float(items["tau"][1]), 1e-6)
+        self.assertEqual(int(items["n"][1]), 42)
+
+    def test_numpy_scalar_state_times(self):
+        """Test that numpy scalar times do not end up as np.float64(...) in the XML."""
+        import numpy as np
+        e = Experiment()
+        e.wait(np.float64(1e-6))
+        e.state_start(np.float64(2e-6))
+        xml = e.write_xml_string()
+        self.assertNotIn("np.float64", xml)
+        self.assertIn('<state time="1e-06"/>', xml)
+        self.assertIn('<state time="2e-06">', xml)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_ExperimentHandling.py

@@ -0,0 +1,70 @@
+import sys
+import threading
+import unittest
+
+from damaris.gui.ExperimentHandling import ExperimentHandling
+from damaris.gui.ResultHandling import ResultHandling
+
+
+class _FakeWriter:
+    def __init__(self):
+        self.sent = []
+
+    def send_next(self, job, quit=False):
+        self.sent.append(job)
+
+
+class _FakeResultIterator:
+    def __init__(self):
+        self.quit_flag = threading.Event()
+
+    def __iter__(self):
+        return iter([])
+
+
+class TestSandboxStdoutRestore(unittest.TestCase):
+    """
+    sys.stdout/sys.stderr are process-global: the sandbox redirect in
+    run() must be undone even when the script raises or returns early.
+    """
+
+    def test_experiment_stdout_restored_after_script_exception(self):
+        orig_stdout = sys.stdout
+        orig_stderr = sys.stderr
+        handled = ExperimentHandling("raise ValueError('boom')", None, None,
+                                     log_callback=lambda line: None)
+        handled.run()
+        self.assertIs(sys.stdout, orig_stdout)
+        self.assertIs(sys.stderr, orig_stderr)
+        self.assertIsNotNone(handled.raised_exception)
+
+    def test_experiment_stdout_restored_on_success(self):
+        orig_stdout = sys.stdout
+        writer = _FakeWriter()
+        handled = ExperimentHandling("print('hello')\nexperiment = lambda: iter([])",
+                                     writer, None, log_callback=lambda line: None)
+        handled.run()
+        self.assertIs(sys.stdout, orig_stdout)
+        self.assertIsNone(handled.raised_exception)
+
+    def test_result_stdout_restored_after_script_exception(self):
+        orig_stdout = sys.stdout
+        orig_stderr = sys.stderr
+        handling = ResultHandling("raise ValueError('boom')", _FakeResultIterator(),
+                                  None, log_callback=lambda line: None)
+        handling.run()
+        self.assertIs(sys.stdout, orig_stdout)
+        self.assertIs(sys.stderr, orig_stderr)
+        self.assertIsNotNone(handling.raised_exception)
+
+    def test_result_stdout_restored_without_result_function(self):
+        orig_stdout = sys.stdout
+        handling = ResultHandling("x = 1", _FakeResultIterator(), None,
+                                  log_callback=lambda line: None)
+        handling.run()
+        self.assertIs(sys.stdout, orig_stdout)
+        self.assertIsNone(handling.raised_exception)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_MeasurementResult.py

@@ -0,0 +1,86 @@
+import io
+import os
+import tempfile
+import unittest
+
+import numpy as np
+import tables
+
+from damaris.data.MeasurementResult import MeasurementResult, read_from_hdf
+
+
+class TestMeasurementResult(unittest.TestCase):
+    def test_add_zero_preserves_data(self):
+        """mr + 0 must return a MeasurementResult with identical data."""
+        mr = MeasurementResult("test_quantity")
+        for i in range(3):
+            mr[i] = float(i * i)
+        mr2 = mr + 0
+        self.assertIsInstance(mr2, MeasurementResult)
+        self.assertEqual(mr2.quantity_name, "test_quantity")
+        np.testing.assert_array_equal(mr2.get_xdata(), mr.get_xdata())
+        np.testing.assert_array_equal(mr2.get_ydata(), mr.get_ydata())
+
+    def test_add_zero_returns_independent_copy(self):
+        """Changing the copy must not change the original."""
+        mr = MeasurementResult("q")
+        mr[1.0] = 5.0
+        mr2 = mr + 0
+        self.assertIsNot(mr2[1.0], mr[1.0])
+        mr2[1.0] = mr2[1.0] + 100.0
+        self.assertEqual(mr[1.0].mean(), 5.0)
+
+    def test_add_nonzero_raises(self):
+        mr = MeasurementResult("q")
+        self.assertRaises(Exception, mr.__add__, 1)
+
+    def test_get_lineplotdata_returns_means(self):
+        """get_lineplotdata must return the accumulated means, not self.y."""
+        mr = MeasurementResult("test_quantity")
+        for i in range(3):
+            mr[float(i)] = float(i * i)
+        k, v = mr.get_lineplotdata()
+        self.assertIsInstance(k, np.ndarray)
+        self.assertIsInstance(v, np.ndarray)
+        self.assertEqual(k.shape, v.shape)
+        np.testing.assert_array_equal(v, mr.get_ydata())
+        np.testing.assert_array_equal(v, [0.0, 1.0, 4.0])
+
+    def test_get_lineplotdata_empty(self):
+        """get_lineplotdata of an empty result must return empty arrays."""
+        mr = MeasurementResult("q")
+        k, v = mr.get_lineplotdata()
+        self.assertEqual(k.shape[0], 0)
+        self.assertEqual(v.shape[0], 0)
+
+    def test_write_to_csv(self):
+        mr = MeasurementResult("q")
+        mr[1.0] = 5.0
+        mr[2.0] = 7.0
+        buf = io.StringIO()
+        mr.write_to_csv(buf)
+        lines = buf.getvalue().splitlines()
+        self.assertEqual(lines[0], "# quantity:q")
+        self.assertEqual(lines[1], "# x y ysigma n")
+        self.assertEqual(len(lines), 4)
+        self.assertEqual(lines[2].split()[3], "1")
+        self.assertEqual(lines[3].split()[3], "1")
+
+    def test_write_to_hdf_roundtrip(self):
+        mr = MeasurementResult("q")
+        mr[1.0] = 5.0
+        mr[2.0] = 7.0
+        with tempfile.TemporaryDirectory() as tmpdir:
+            path = os.path.join(tmpdir, "test.h5")
+            with tables.open_file(path, mode="w") as h5file:
+                mr.write_to_hdf(h5file, "/", "mr", "test result")
+            with tables.open_file(path, mode="r") as h5file:
+                mr2 = read_from_hdf(h5file.get_node("/mr"))
+        self.assertIsNotNone(mr2)
+        self.assertEqual(mr2.quantity_name, "q")
+        np.testing.assert_array_equal(mr2.get_xdata(), mr.get_xdata())
+        np.testing.assert_array_equal(mr2.get_ydata(), mr.get_ydata())
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_Signalpath.py

@@ -0,0 +1,97 @@
+import unittest
+from datetime import datetime
+
+import numpy as np
+from damaris.data.ADC_Result import ADC_Result
+
+
+def rotate_reference(y, degrees):
+    "reference rotation of the two channels by degrees"
+    rad = np.deg2rad(degrees)
+    cos, sin = np.cos(rad), np.sin(rad)
+    return [cos * y[0] - sin * y[1], sin * y[0] + cos * y[1]]
+
+
+class TestSignalpath(unittest.TestCase):
+    @staticmethod
+    def create_adc_result():
+        x = np.arange(4, dtype="float32")
+        y = [np.array([10, 20, 30, 40], dtype="int16"),
+             np.array([15, 25, 35, 45], dtype="int16")]
+        index = [(0, 3)]
+        job_date = datetime(2023, 1, 1)
+        desc = {"key": "value"}
+        sampl_freq = 1000.0
+        job_id = 11
+        adc = ADC_Result(x, y, index, sampl_freq, desc, job_id, job_date)
+        adc.set_nChannels(2)
+        return adc
+
+    def assertRotated(self, adc, degrees):
+        rotated = adc.phase(degrees)
+        expected = rotate_reference(adc.y, degrees)
+        np.testing.assert_allclose(rotated.y[0], expected[0])
+        np.testing.assert_allclose(rotated.y[1], expected[1])
+        return rotated
+
+    def test_phase_positive_quarter_turns(self):
+        """Test phase() with 0, 90, 180 and 270 degrees."""
+        adc = self.create_adc_result()
+        for degrees in (0, 90, 180, 270):
+            self.assertRotated(adc, degrees)
+
+    def test_phase_negative_quarter_turns(self):
+        """Test phase() with negative multiples of 90 degrees."""
+        adc = self.create_adc_result()
+        for degrees in (-90, -180, -270, -360):
+            self.assertRotated(adc, degrees)
+
+    def test_phase_negative_arbitrary_angles(self):
+        """Test phase() with arbitrary (not multiple of 90) negative angles."""
+        adc = self.create_adc_result()
+        for degrees in (-45, -120, -315):
+            self.assertRotated(adc, degrees)
+
+    def test_phase_equals_equivalent_positive_angle(self):
+        """Test that negative angles give the same result as the equivalent positive angle."""
+        adc = self.create_adc_result()
+        for degrees in (-90, -45, -120):
+            np.testing.assert_allclose(adc.phase(degrees).y,
+                                       adc.phase(degrees + 360).y)
+
+    def test_phase_does_not_modify_original(self):
+        """Test that phase() leaves the original data (and its arrays) untouched."""
+        adc = self.create_adc_result()
+        original = [adc.y[0].copy(), adc.y[1].copy()]
+        for degrees in (0, 90, 180, 270, -90, -45, 45):
+            rotated = adc.phase(degrees)
+            self.assertIsNot(rotated, adc)
+            self.assertIsNot(rotated.y, adc.y)
+            for ch in (0, 1):
+                self.assertIsNot(rotated.y[ch], original[ch])
+                np.testing.assert_array_equal(adc.y[ch], original[ch])
+
+    def test_phase_result_independent(self):
+        """Test that modifying the phase-rotated copy does not affect the original."""
+        adc = self.create_adc_result()
+        rotated = adc.phase(-90)
+        rotated.y[0][:] = 9999
+        np.testing.assert_array_equal(adc.y[0], [10, 20, 30, 40])
+
+    def test_phase_chaining(self):
+        """Test that chained phase() calls add up correctly."""
+        adc = self.create_adc_result()
+        np.testing.assert_allclose(adc.phase(-90).phase(-90).y,
+                                   adc.phase(-180).y)
+        np.testing.assert_allclose(adc.phase(-90).phase(90).y,
+                                   adc.phase(0).y)
+
+    def test_phase_wrong_channel_number(self):
+        """Test the Exception raised for signalpaths with other than 2 channels."""
+        adc = self.create_adc_result()
+        adc.y.pop()
+        self.assertRaises(Exception, adc.phase, -90)
+
+
+if __name__ == "__main__":
+    unittest.main()