[Octave-bug-tracker] [bug #50974] SEGFAULT with constructed cmdline_options

[Steven Bergner]

URL:
  <http://savannah.gnu.org/bugs/?50974>

                 Summary: SEGFAULT with constructed cmdline_options
                 Project: GNU Octave
            Submitted by: steb
            Submitted on: Mon 08 May 2017 09:12:17 PM UTC
                Category: Interpreter
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Segfault, Bus Error, etc.
                  Status: None
             Assigned to: None
         Originator Name: Steven
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 4.2.1
        Operating System: Any

    _______________________________________________________

Details:

There is a potential for SEGFAULT when using Octave as library via embedded
interpreter inside an application that has its own command line arguments,
independent of the arguments that are passed to some internal
octave::cli_application instance.

The bug reaches back a number of versions and is still present in v4.3.0+.

In file libinterp/octave.cc
the function
cmdline_options::cmdline_options (int argc, char **argv)
constructs an object like this
m_remaining_args = string_vector (argv+octave_optind_wrapper (),
argc-octave_optind_wrapper ())

Here, argv may be a user-constructed char**, whereas octave_optind_wrapper()
returns the optind integer from getopt.h, which reports the number of command
line arguments of the actual application.

When using Octave as a library, this constructed address may point to invalid
memory and a SEGFAULT is triggered.

Observation and mitigation of this bug is documented here:
https://github.com/renozao/RcppOctave/pull/17/commits/7cdb1d4f0b6253920869ac74709554fc822cac96
related commit msg: "
- octave.cc (v4.2, 4.3+) cmdline_options::cmdline_options() uses
  argv + octave_optind_wrapper(), where argv is the char** produced
  in rcpp_octave.cpp octave_session and optind is a global int
  giving the number of arguments getopt.h reports for the executable,
  having nothing to do with the args produced here. If the executable
  was called with more than narg arguments, this gives a SEGFAULT.
  resolved: octave_session now ensures that char** is long enough
"
with corresponding code changes in function octave_session in
src/rcpp_octave.cpp of https://github.com/git-steb/RcppOctave/tree/develop

Thanks for keeping up this great project!

Steven
https://github.com/git-steb





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?50974>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/