SGI: Security

As we're talking about our favourite shells, I used to be a zsh adherent after abandoning linux, but I've recently gone to pdksh and feel quite comfortable. zsh went out the door when they started doing "easy install" things on first login.

duck wrote: zsh went out the door when they started doing "easy install" things on first login.

jus stick to an older version then

foetz wrote:

duck wrote: zsh went out the door when they started doing "easy install" things on first login.

jus stick to an older version then

Hah, yes! Suggested in this of all threads

duck wrote:

foetz wrote:

duck wrote: zsh went out the door when they started doing "easy install" things on first login.

jus stick to an older version then

Hah, yes! Suggested in this of all threads

hehe, no prob if it's not your webserver unless you have other users on it you don't trust. given that the zsh version you like does have nasty security issues at all

foetz wrote:

duck wrote: Hah, yes! Suggested in this of all threads

hehe, no prob if it's not your webserver unless you have other users on it you don't trust. given that the zsh version you like does have nasty security issues at all

Of course. It's not like it's replacing sh.

The ksh source was opened by AT&T maybe five or six years ago. Before that it was free for end-user download since the end of the '90s approximately, though the build environment was not entirely friendly.

kjaer wrote: though the build environment was not entirely friendly.

oh what an understatement

Pretty early on I read the essay, "Csh Programming Considered Harmful," so I never bothered learning csh or tcsh. Since I was using Linux, bash was the default shell typically, and it seemed to do interactive editing and scripting pretty well.

Later when I was doing a lot of shell scripting on HP-UX, I used ksh88, and I really liked that as well. It has the important things that bash has, but without the bloat. When I was using it, though, there were some really annoying compatibility issues between ksh88 and pdksh. With ksh88, the following script prints "1", and on pdksh and mksh, it prints "0". Bash also prints "0".


It's something stupid related to pipes and processes. The programmer needs to know that anything happening in the body of a loop is happening in another context -- but only when something is being piped into the loop. Why this behavior is reasonable, I have no idea. ksh88 handles it just fine, and did so decades ago. I don't know why these other shells like pdksh, mksh, and bash put the burden of remembering arcane details like this onto the programmer.

When AT&T opened up ksh93, I wish they had also released ksh88. My impression has been that ksh88 is a good all-around shell. It doesn't hurt that it's a long-time standard on commercial Unix systems either.

jwp wrote: Pretty early on I read the essay, "Csh Programming Considered Harmful," so I never bothered learning csh or tcsh. Since I was using Linux, bash was the default shell typically, and it seemed to do interactive editing and scripting pretty well.

Later when I was doing a lot of shell scripting on HP-UX, I used ksh88, and I really liked that as well. It has the important things that bash has, but without the bloat. When I was using it, though, there were some really annoying compatibility issues between ksh88 and pdksh. With ksh88, the following script prints "1", and on pdksh and mksh, it prints "0". Bash also prints "0".

Code: Select all

x=0
echo onetime | while read line; do
x=1
done
echo $x

It's something stupid related to pipes and processes. The programmer needs to know that anything happening in the body of a loop is happening in another context -- but only when something is being piped into the loop. Why this behavior is reasonable, I have no idea. ksh88 handles it just fine, and did so decades ago. I don't know why these other shells like pdksh, mksh, and bash put the burden of remembering arcane details like this onto the programmer.

When AT&T opened up ksh93, I wish they had also released ksh88. My impression has been that ksh88 is a good all-around shell. It doesn't hurt that it's a long-time standard on commercial Unix systems either.

That's an easy one - variable scope. Consider the following C++ program



The variable inside of the loop expires when the loop ends, and is in a different scope then the variable outside of the loop, despite having the same name. My guess is the developers of the other shells felt that variable scope was important, where as ksh88 doesn't have the concept.

armanox wrote: That's an easy one - variable scope. [...] The variable inside of the loop expires when the loop ends, and is in a different scope then the variable outside of the loop, despite having the same name. My guess is the developers of the other shells felt that variable scope was important, where as ksh88 doesn't have the concept.

Ah, but this only happens when piping to a loop . Otherwise, all the shells act the same way with variables, loops, and scope. The inconsistency is with pdksh, mksh, and bash. If the script were to be rewritten this way, then it would work the exact same way on all shells:


This sort of incompatibility means that pdksh and mksh cannot be used as serious replacements for ksh. The only real path for developing or running ksh88 scripts with open-source software is to try them under ksh93, which is more compatible (and follows ksh88 behavior for this pipe / loop stuff).

Most people happily use pdksh and mksh as ksh replacements, because they don't do a lot of shell scripting, or otherwise don't have to worry about shell scripting compatibility. Sadly, it seems that many shells follow the same behavior as pdksh and mksh, despite there being no rationale other than the implementation details of the shell.

jwp wrote:

armanox wrote: That's an easy one - variable scope. [...] The variable inside of the loop expires when the loop ends, and is in a different scope then the variable outside of the loop, despite having the same name. My guess is the developers of the other shells felt that variable scope was important, where as ksh88 doesn't have the concept.

Ah, but this only happens when piping to a loop . Otherwise, all the shells act the same way with variables, loops, and scope. The inconsistency is with pdksh, mksh, and bash. If the script were to be rewritten this way, then it would work the exact same way on all shells:

Code: Select all

x=0
echo onetime > /tmp/onetime
while read line; do
x=1
done < /tmp/onetime
echo $x

This sort of incompatibility means that pdksh and mksh cannot be used as serious replacements for ksh. The only real path for developing or running ksh88 scripts with open-source software is to try them under ksh93, which is more compatible (and follows ksh88 behavior for this pipe / loop stuff).

Most people happily use pdksh and mksh as ksh replacements, because they don't do a lot of shell scripting, or otherwise don't have to worry about shell scripting compatibility. Sadly, it seems that many shells follow the same behavior as pdksh and mksh, despite there being no rationale other than the implementation details of the shell.

Interesting. I'll admit that shell scripting is not an area that I consider myself an expert in. I always considered the lack of variable declaration a potential source of confusion in a lot of scripting languages (in my C++ example, it's clear that I am declaring a new i in the for loop. If I do not declare i, it uses the variable from outside of the loop's scope (so for( i = 0; i < 2; i++)) rather then using a local one).

armanox wrote: That's an easy one - variable scope. Consider the following C++ program

Code: Select all


#include <iostream>
using namespace std;

int main( int argc, char* argv[])
{
int i = 2;
for (int i = 0; i < 2; i++)
cout << i << endl; // Prints 0 then 1
cout << i << endl; // Prints 2
return 0;
}



.

declaring i twice, putting in args that are never used and using c++ for something that trivial at all?

just pulling your leg, the point was clear anyway

foetz wrote:

armanox wrote: That's an easy one - variable scope. Consider the following C++ program

Code: Select all


#include <iostream>
using namespace std;

int main( int argc, char* argv[])
{
int i = 2;
for (int i = 0; i < 2; i++)
cout << i << endl; // Prints 0 then 1
cout << i << endl; // Prints 2
return 0;
}



.

declaring i twice, putting in args that are never used and using c++ for something that trivial at all?

just pulling your leg, the point was clear anyway

Declaring i twice was needed for the demo The args are a habit, even if they aren't needed. And I happen to like C++, thank you very much. :p

Not to be picayune, but dumping std into the global namespace is considered harmful...



Okay, so I am being picayune...

haha, poor armanox

but since we're on it already why messing with c++ at all?

this is a classic "led down the garden path" situation, which you do need to have some programming ability to notice.

you concluded that "It's something stupid related to pipes and processes"; in other words, that the statement "x=1" was not affecting the value of x, because it (surprisingly) executes in a different process. this is an unwarranted assumption, since it might not have affected x simply by never executing at all.


ls: quux: No such file or directory

for those who "do a lot of shell scripting" and are led astray by such basic mistakes, the greater danger may not be their choice of shell, but letting them near computers to begin with.


is a newline, because line is empty.


and

and
work as expected. the read runs without spawning a subshell; where it differs from other shells is in its handling of stdin: by default, if stdin is not a terminal, read doesn't wait for its input, but returns immediately if there is no data.

robespierre wrote: this is a classic "led down the garden path" situation, which you do need to have some programming ability to notice.

you concluded that "It's something stupid related to pipes and processes"; in other words, that the statement "x=1" was not affecting the value of x, because it (surprisingly) executes in a different process. this is an unwarranted assumption, since it might not have affected x simply by never executing at all.

It might be an unwarranted assumption, except for the fact that I tested this pretty carefully years ago, and then again recently.

robespierre wrote:

Code: Select all

x=0
echo onetime | while read line; do
touch quux
done
ls quux

ls: quux: No such file or directory

for those who "do a lot of shell scripting" and are led astray by such basic mistakes, the greater danger may not be their choice of shell, but letting them near computers to begin with.

Sorry, but your script results are wrong using all the shells I'm talking about. All of them create the file quux, and then list it successfully, as you can see here:


So you posted bogus results, and then sneered that I shouldn't even be allowed near a computer? Okay... if you want an even simpler example:


The shell loops work just fine.

robespierre wrote:

Code: Select all

echo foo bar | read line; echo $line

is a newline, because line is empty.

Yeah, that's why the example scripts I gave never piped to "read", but rather to "while read", like this:


So you didn't bother to notice the difference, yet you're the one with the "real programming ability" who is able to spot these things. Yeah, okay...

One last script to illustrate the incompatibility:


I'm not some big shell scripting guru, but I'm also not a bumbling idiot who just makes this stuff up because I just lack "real programming ability." I ran into this particular problem because real shell scripts I was writing for HP-UX servers would fail in Cygwin because pdksh and mksh are not faithful ksh88 clones. This sort of incompatibility is dangerous because there is no indication of it other than getting the wrong (old) variable values. Other people have run into it as well, usually when they try to migrate their ksh88 scripts to Linux, and then run into all sorts of errors.

Those tests were all done using "sh (AT&T Labs Research) 1993-12-28 p". the point is that even "ksh93" does not run the example as expected.

I think that the behavior of bash/dash is informed by POSIX.2:

Additionally, each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment.

They interpret it to require that the behavior as respects variable assignment be as if the command was executed in a subshell, even though it is not.