If I understood your previous email correctly, and the issue you mention here it sounds like you need to do some input checking and preconditioning before processing. I'll try to take a look at your code later today, but for the case of the third input either being a vector or a string you would want to have octave check the class of the input for that variable prior to doing any computation. Since there is a variable number of possible inputs, you will want to use 'varargin' to collect them and then do type checking to decide what to do with them and how to run the code. Again, the quadgk function is a good example of this as it has to deal with both a variable number of possible inputs and mixed numerical and string inputs. There is also now a supported inputParser class that supposedly is a more elegant way to do this, but I'm not familiar with it so I just do it the old-fashioned way.
Then for any vectors, if they don't come in the orientation you want, you can have octave change them to either a row or column inside the function as you desire. I find the fastest way to do this is either
A = A(:)
for column output or
A = A(:)'
for row output