CentOS Bug Tracker
CentOS Website
Main | My View | View Issues | Roadmap | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ View Advanced ] [ Issue History ] [ Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0002803 [CentOS-5] ksh major always 2008-04-22 19:10 2009-02-18 16:44
Reporter serge_bonin View Status public  
Assigned To
Priority normal Resolution open  
Status acknowledged   Product Version 5.0 - x86_64
Summary 0002803: getopts function does not work properly when used in a function that is invoked with back quotes
Description The getopts function does not seem to work properly when used in a function that is invoked with back quotes. The OPTIND variable is not incremented on subsequent calls to the getopts function; only the first call works properly.

Let's say we define a function using getopts as follows:

#!/bin/ksh

test_function()
{
   typeset COMM_LCL_OPTION

   OPTIND=1
   while getopts :ci: COMM_LCL_OPTION; do
      case ${COMM_LCL_OPTION} in
         c)
            echo " *** Function received argument -c"
            ;;
         i)
            echo " *** Function received argument -i ${OPTARG}"
            ;;
      esac
   done
   echo " OPTIND: ${OPTIND}"
   echo " ARGS: ${*}"
   shift $((${OPTIND} - 1))

   echo " Any remaining argument in function call: ${*}"
}


If this function is called as follows:

echo "Executing function:"
echo " First call"
RET=`test_function -c -i arg1 -iarg2 call1`
echo "$RET"
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"

The first call will execute properly but all subsequent will not increment the OPTIND variable as it should:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: -c -i arg1 -iarg2 call2

If this function is called directly instead of using back quotes, the results are in line with what is expected:

echo "Executing function directly:"
echo " First call"
test_function -c -i arg1 -iarg2 call1
echo "$RET"
echo " Second call"
test_function -c -i arg1 -iarg2 call2
echo "$RET"

As seen in the output, the OPTIND variable is properly incremented:

Executing functioni directly:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2

However, if a call is made to the function with back quotes first and then directly to the function:

echo "Executing function:"
echo " First call"
RET=`test_function -c -i arg1 -iarg2 call1`
echo "$RET"
echo " Second call"
test_function -c -i arg1 -iarg2 call2
echo "$RET"

The getopts function does not work properly on subsequent calls, even if these calls are directly done to the function:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: -c -i arg1 -iarg2 call2

However, this does not seem to affect sub scripts that are called after a back quoted function:

echo "Executing sub program:"
echo " First call"
RET=`test_function -c -i arg1 -iarg2 call1`
echo "$RET"
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"
echo " Third call"
RET=`./check_sub_getopts.ksh -c -i arg1 -iarg2 call3`
echo "$RET"

The check_sub_getopts.ksh script contains the exact same code as the test_function function stated above. The result is as follows:

Executing sub program:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: -c -i arg1 -iarg2 call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
Additional Information
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0007166)
serge_bonin (reporter)
2008-04-22 19:48

 I forgot to add that this script was tested on OSF1, AIX, Solaris, CentOS version 4, and Unix services for Windows. On all of these OSs the getopts functions would increment the OPTIND variable correctly (its value would go up to 5 on each call to the test_function using back quotes or calling it directly).

After further testing, I had discovered that it is not only the OPTIND variable that is not properly incremented, but the getopts function does not even detect arguments anymore. This can be seen with the following calls:

echo "Executing function:"
echo " First call"
test_function -c -i arg1 -iarg2 call1
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"
echo " Third call"
test_function -c -i arg1 -iarg2 call3
echo " Forth call"
RET=`test_function -c -i arg1 -iarg2 call4`
echo "$RET"
echo " Fifth call"
RET=`./check_sub_getopts.ksh -c -i arg1 -iarg2 call5`
echo "$RET"

The output is:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: -c -i arg1 -iarg2 call3
   Forth call
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: -c -i arg1 -iarg2 call4
   Fifth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call5
      Any remaining argument in function call: call5


We can see that in the third and forth call, no matter if the function was called directly or with back quotes, the getopts function does not even detected the presence of the -c and -i arguments. Good news is that at least the external shell has the correct getopts info (fifth call).

If I run the exact same script in bash, I do not get the same result:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4
   Fifth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call5
      Any remaining argument in function call: call5

As we can see, in bash, the OPTIND and arguments seem to be processed correctly.
(0007167)
range (administrator)
2008-04-23 08:44

 You should report this upstream at http://bugzilla.redhat.com/ [^] as this can only be fixed there - meaning: We won't do a different build from what upstream is building. But thanks for reporting - and please leave a reference to the bug in here.

Cheers,

Ralph
(0007171)
serge_bonin (reporter)
2008-04-23 20:29

 Thanks Ralph,

I also discovered a second bug today which might be related to what I see with getopts (local and global variables):

I use a lot of local variable in my function in ksh. This is done by using typeset. In this version of ksh, local variables do not seem to work any more. This simple program shows this:

sub_func()
{
   typeset ATTRIBUT=6

   echo "The function set a private variable to : $ATTRIBUT"
}

typeset ATTRIBUT=1
echo "The following should have the same value before and after a function call:"
echo "Before function call: $ATTRIBUT"
sub_func
echo "After function call: $ATTRIBUT"

If I execute it on centos version 4 (and on AIX, OSF1, Solaris, HP-UX), I get the following result:

Before function call: 1
The function set a private variable to : 6
After function call: 1


If I execute on my latest version of centos, I get:

The following should have the same value before and after a function call:
Before function call: 1
The function set a private variable to : 6
After function call: 6

As you can see, the local variable definition in the sub funciton is no longer respected by ksh.

I will report this to Red Hat as well... and stop using ksh...
(0007173)
serge_bonin (reporter)
2008-04-24 20:07

 This is Redhat's response to my local variable issue that I report in a sub note to this bug (note 7171):

I don't think that this is a bug.

The ksh manual page says that functions defined with 'function name' syntax
share the variables with the calling program with an exception of variables
defined with 'typeset' command that are defined as local in the scope of the
function.

Things are different with functions defined with 'name()' syntax. Those
functions run completely in the caller's environment (no exception for typeset).
 This also explains the "issue" in comment 0000001 -- try to define the function as
'function sub_func'... This is a documented and expected behaviour.

I think the biggest problem here is that the behaviour was different in ksh88
(RHEL-5 comes with ksh93) that is present on AIX, HP-UX, old Solaris (I think
the new OpenSolaris is already shipped with ksh93 as well), etc. which causes
the confusion.
(0007174)
serge_bonin (reporter)
2008-04-24 20:09

 This is the reponse for the getopts issue that I was having:

There is one thing I don't understand in the function body:

   OPTIND=1

What's this line good for? OPTIND is set automatically by the shell...

The behaviour is otherwise quite interesting... I don't know if it's a bug or
if changing OPTIND value explicitly couldn't erase it's special meaning
(unsetting it does).
(0007175)
serge_bonin (reporter)
2008-04-24 20:11

 From the response to the local variable and getopts issues, I made additional tests and found the following:

Thanks to your help in my issue reported in bug #431448, the problem goes away
when I use the 'function test_function' format instead of 'test_function()'.

I believe this changes the scope of the variables used by getopts in the
function. This used to work in ksh88 but it changed in ksh93.

I did have to remove the OPTIND=1 in the test_function for this to work under
ksh. However, the reason I had it there in the first place, which used to work
under ksh88, is that bash does not work properly if it isn't there.

I can demonstrate this, if a call the test_function in this sequence:

echo "Executing function:"
echo " First call"
test_function -c -i arg1 -iarg2 call1
echo " Second call"
RET=`test_function -c -i arg1 -iarg2 call2`
echo "$RET"
echo " Third call"
RET=`test_function -c -i arg1 -iarg2 call3`
echo "$RET"
echo " Forth call"
test_function -c -i arg1 -iarg2 call4

This is the result of ksh using the 'function test_function' format without
OPTIND:

The following should always give the same result:
Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4

We now see that the getopts function is reacting normally on multiple calls.

If I run the same thing with bash, I get:

The following should always give the same result:
Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4

As we see, OPTIND seems to be set to the value of the previous call to getopts.
If I put back the OPTIND=1 statement before the getopts function and re-execute
it in bash, I get the expected result:

Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: call4

If I execute the modified bash script in ksh, I don’t get the same result:

The following should always give the same result:

The following should always give the same result:
Executing function:
   First call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call1
      Any remaining argument in function call: call1
   Second call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 5
      ARGS: -c -i arg1 -iarg2 call2
      Any remaining argument in function call: call2
   Third call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call3
      Any remaining argument in function call: -c -i arg1 -iarg2 call3
   Forth call
      *** Function received argument -c
      *** Function received argument -i arg1
      *** Function received argument -i arg2
      OPTIND: 1
      ARGS: -c -i arg1 -iarg2 call4
      Any remaining argument in function call: -c -i arg1 -iarg2 call4

For some reason, ksh increments the value of OPTIND in the first and second
call but stops doing it in the third and forth call to the function.

If I put a typeset to the OPTIND variable in the test_function, none of the
calls work in ksh but the all work well under bash.


I just moved all of my scripts from ksh88 to bash as bash seems to be more
compatible with the ‘mistakes’ I did in ksh88 (getopts and typeset local
variables in function() name format) in respect to what is now done in ksh93.
Do you believe that bash will also move in the direction that ksh93 has gone to
or will bash continue to behave in this manner?
(0008741)
range (administrator)
2009-02-18 16:44

 ksh will be completely rebased in 5.3 - please wait for that and try again.

- Issue History
Date Modified Username Field Change
2008-04-22 19:10 serge_bonin New Issue
2008-04-22 19:10 serge_bonin Assigned To => kbsingh@karan.org
2008-04-22 19:48 serge_bonin Note Added: 0007166
2008-04-23 08:44 range Note Added: 0007167
2008-04-23 08:44 range Assigned To kbsingh@karan.org => range
2008-04-23 08:44 range Status new => acknowledged
2008-04-23 20:29 serge_bonin Note Added: 0007171
2008-04-24 20:07 serge_bonin Note Added: 0007173
2008-04-24 20:09 serge_bonin Note Added: 0007174
2008-04-24 20:11 serge_bonin Note Added: 0007175
2009-02-18 16:44 range Note Added: 0008741

Copyright © 2000 - 2009 Mantis Group
Powered by Mantis Bugtracker