[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of
From: |
Arun Giridhar |
Subject: |
[Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data |
Date: |
Mon, 13 Feb 2023 19:38:57 -0500 (EST) |
URL:
<https://savannah.gnu.org/bugs/?63803>
Summary: Saving causes OOM, crash, and loss of already saved
data
Project: GNU Octave
Submitter: arungiridhar
Submitted: Mon 13 Feb 2023 07:38:55 PM EST
Category: Interpreter
Severity: 4 - Important
Priority: 5 - Normal
Item Group: Segfault, Bus Error, etc.
Status: None
Assigned to: None
Originator Name:
Originator Email:
Open/Closed: Open
Release: dev
Discussion Lock: Any
Operating System: GNU/Linux
Fixed Release: None
Planned Release: 9.1.0 (current default)
_______________________________________________________
Follow-up Comments:
-------------------------------------------------------
Date: Mon 13 Feb 2023 07:38:55 PM EST By: Arun Giridhar <arungiridhar>
This is on: GNU Octave Version: 9.0.0 (hg id: 3e4e74ad8fd7)
Prep work: Create a few arrays, save to file, and exit Octave:
adj = ~eye(36);
edges = reshape (1:120, 60, 2);
lst = false (5735500, 60);
ncells = 36;
nedges = 60;
whos
save -binary test.mat
Output is normal:
octave:6> whos
Variables visible from the current scope:
variables in scope: top scope
Attr Name Size Bytes Class
==== ==== ==== ===== =====
adj 36x36 1296 logical
edges 60x2 960 double
lst 5735500x60 344130000 logical
ncells 1x1 8 double
nedges 1x1 8 double
Total is 344131418 elements using 344132272 bytes
octave:7> save -binary test.mat
octave:8> exit
Then restart Octave, load that file, then create a new large variable, and try
to save everything to the same file. I have 16 GB RAM, and I can comfortably
create a 7.5 GB array with no second thought:
load test.mat
whos
arr = false (36, 36, rows(lst));
arr(:, :, 1:2:end) = repmat(mod((1:36)' + (1:36), 2) == 1, 1, 1,
size(arr,3)/2);
arr(:, :, 2:2:end) = ~arr(:, :, 1:2:end);
whos
save -binary test.mat
Output:
octave:6> whos
Variables visible from the current scope:
variables in scope: top scope
Attr Name Size Bytes Class
==== ==== ==== ===== =====
adj 36x36 1296 logical
arr 36x36x5735500 7433208000 logical
edges 60x2 960 double
lst 5735500x60 344130000 logical
ncells 1x1 8 double
nedges 1x1 8 double
Total is 7777339418 elements using 7777340272 bytes
octave:7> save -binary test.mat
Killed
What happened?! Let's restart Octave and see what happened:
octave:1> load test.mat
octave:2> whos
Variables visible from the current scope:
variables in scope: top scope
Attr Name Size Bytes Class
==== ==== ==== ===== =====
adj 36x36 1296 logical
Total is 1296 elements using 1296 bytes
All the other variables have been lost both from RAM *and from the saved
file*.
This actually happened to me in the last 30 minutes. Fortunately I lost only
12 minutes of work and was able to rerun a script that regenerated everything.
Had it been more important data this could have been BAD!
Raising importance because of the ease and surprise of data loss right in the
act of saving it, even with plenty of memory remaining.
Observations if it helps anyone localize the cause:
* Monitoring the memory usage through top, Octave somehow uses 16 GB RAM + 8
GB swap, runs out of memory, and is killed. It's not clear why the memory use
goes that high for a total of less than 8 GB worth of variables. Looks like
copy-on-write happening where it shouldn't.
* The saving works as intended if the new array "arr" is all-false. But if
it's set to a checkerboard pattern like in the case above, it crashes while
saving. Not sure why the density of the array causes a crash since it's not
even a sparse object, but it empirically does. Maybe something to do with the
way it's created? Copy-on-write problem with ranges?
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?63803>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data,
Arun Giridhar <=
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Nicholas Jankowski, 2023/02/13
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Rik, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, John W. Eaton, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Arun Giridhar, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, John W. Eaton, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Arun Giridhar, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Arun Giridhar, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Arun Giridhar, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Rik, 2023/02/14
- [Octave-bug-tracker] [bug #63803] Saving causes OOM, crash, and loss of already saved data, Rik, 2023/02/14