Training, Open Source computer languages
PerlPHPPythonMySQLApache / TomcatTclRubyJavaC and C++LinuxCSS 
Search for:
Home Accessibility Courses Diary The Mouth Forum Resources Site Map About Us Contact
 
For 2021 - online Python 3 training - see ((here)).

Our plans were to retire in summer 2020 and see the world, but Coronavirus has lead us into a lot of lockdown programming in Python 3 and PHP 7.
We can now offer tailored online training - small groups, real tutors - works really well for groups of 4 to 14 delegates. Anywhere in the world; course language English.

Please ask about private 'maintenance' training for Python 2, Tcl, Perl, PHP, Lua, etc.
Expect --> Bash --> Script

Posted by ShipConner (ShipConner), 22 June 2005
Howdy all  

Is it possible to spawn a bash shell, send it a script to execute, and have expect control the programs that get started in the script? Here is some example code:
--------------------------
spawn /bin/bash

send -- "/scripts/mysshscript\r"
expect {
       timeout {puts $logg_file "ERROR timeout"}
       eof {puts $logg_file "ERROR eof"}
       -exact "continue connecting (yes/no)? " {send -- "yes\r"; exp_continue}
       -exact "assword:" {send -- "$password\r"; exp_continue}
       -re "$prompt" {set cmd_result "$expect_out(buffer)"}
}
# end of expect cmd results
----------------------------------

mysshscript looks like this:
--------------------------------
for i in $(cat serversfile)
do
ssh $i
hostname
done
--------------------------------

I know expect can retain control when I spawn bash and send ssh commands directly to bash but I would like to have expect control programs started by bash... has anyone had any success with this?

From my experiments it would seem bash *not expect has direct control of the i/o for the programs started in the script.

Thanks in advance Graham  

Posted by admin (Graham Ellis), 22 June 2005
Anything you can do from the command line you should be able to do from expect (except pour coffee into the keyboard   ) ... so that means that you can send up a script into a file (perhaps using ftp or cat to do the transfer?) then execute it.



Posted by ShipConner (ShipConner), 22 June 2005
I am confused. When I execute the above code expect won't respond to the password: prompt that I get from ssh.

expcript-->spawns bash
send "please exe this bashscript" --> bash (i.e. bash /myscript)

myscript --> ssh to remote host
<--password

expect -ex "assword" ---> doesn't reply to the password prompt, it just sits there untill I type something



Posted by admin (Graham Ellis), 22 June 2005
OK ... that's a different question to the one I answered  

Educated guess - it suspect that your program IS responding but too quickly.   Passwords are odd things - they're not read from STDIN but from /dev/kbd or similar - that's done to make them harder to automate and hack.  

Firstly, check to see whether the remote system sends a space after the colon of assword, and if it does wait for that space.  Secondly (and even if the first change appears to make no difference), try a little delay before sending the password back.  It's probbaly the password is being sent before the login shell is actually ready for it.

Posted by ShipConner (ShipConner), 22 June 2005
Thank you for the reply.

I am still a little confused. When I debug with "exp_internal 1" I see I am talking to the original process I spawned "exp6" and that it is the one sending me passwd prompt... but my expect -re "assword" won't kick in. I know my syntax is correct becuase when I send the ssh commands directly to bash (rather than have bash exe another bash script) it works perfectly.

www:/scripts # ./xbash
spawn /bin/bash
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {16733}
send: sending "/gdms/scripts/myscript\r" to { exp6 }

expect: does "" (spawn_id exp6) match exact string "continue connecting (yes/no)? "? no
"assword: "? no
"(%|#|>|\$) $"? no
/myscript

expect: does "/myscript\r\n" (spawn_id exp6) match exact string "continue connecting (yes/no)? "? no
"assword: "? no
"(%|#|>|\$) $"? no
www:/scripts #
expect: does "/myscript\r\nwww:/scripts # " (spawn_id exp6) match exact string "continue connecting (yes/no)? "? no
"assword: "? no
"(%|#|>|\$) $"? yes
expect: set expect_out(0,string) "# "
expect: set expect_out(1,string) "#"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "/myscript\r\nwww:/scripts # "
tty_raw_noecho: was raw = 0  echo = 1
spawn id exp6 sent </>
spawn id exp6 sent <m>
mspawn id exp6 sent <y>
yspawn id exp6 sent <s>
sspawn id exp6 sent <c>
cspawn id exp6 sent <r>
rspawn id exp6 sent <i>
ispawn id exp6 sent <p>
pspawn id exp6 sent <t>
tspawn id exp6 sent <\r\n>

spawn id exp6 sent <44.44.44.44\r\n>
44.44.44.44
spawn id exp6 sent <Password: >
Password:
-------------------------------

I also included a pstree to show the relation ship
sshd---bash---myexpect---bash---myscript---ssh

Thanks again for any help.                                




Posted by ShipConner (ShipConner), 22 June 2005
Ok, sorry I didn't see you last post let me experiment with that. THANKS!

Posted by ShipConner (ShipConner), 22 June 2005
Hmmmm... The password did have an extra space after it but adding it didn't help. I also tried slowing it down by three seconds, thats proving frutile as well. I did notice that my expect doesn't time out(set to default).... its almost like the subshell is in control of the ssh channel blocking expect from interacting with it.

She is going to give I can feel it  

Posted by ShipConner (ShipConner), 22 June 2005
Some very ugly code from strace:

write(2, "send: sending \"", 15send: sending ")        = 15
write(2, "/gdms/scripts/myscript\\r", 24/gdms/scripts/myscript\r) = 24
write(2, "\" to {", 6" to {)                  = 6
write(2, " exp6 ", 6 exp6 )                   = 6
write(6, "/gdms/scripts/myscript\r", 23) = 23
write(2, "}\r\n", 3}
)                    = 3
time([1119384141])                      = 1119384141
time([1119384141])                      = 1119384141
write(2, "\r\nexpect: does \"", 16
expect: does ")     = 16
write(2, "\" (spawn_id exp6) match exact st"..., 37" (spawn_id exp6) match exact string ) = 37
write(2, "\"", 1")                       = 1
write(2, "continue connecting (yes/no)? ", 30continue connecting (yes/no)? ) = 30
write(2, "\"? ", 3"? )                     = 3
write(2, "no\r\n", 4no
)                   = 4
write(2, "\"", 1")                       = 1
write(2, "Password:  ", 11Password:  )             = 11
write(2, "\"? ", 3"? )                     = 3
write(2, "no\r\n", 4no
)                   = 4
write(2, "\"", 1")                       = 1
write(2, "(%|#|>|\\$) $", 12(%|#|>|\$) $)           = 12
write(2, "\"? ", 3"? )                     = 3
write(2, "no\r\n", 4no
)                   = 4

I can see expect sent the script to bash to execute. And I can see the password prompt come back but it just sits there.

It is a little after 4:00am I am going to bed. Graham, thank you for all the help. I will report in with any new findings.

Posted by ShipConner (ShipConner), 11 December 2005
Hehe, a little late since my last post.

It seems I was making things too complicated. Just use bash to do everything (Looping, lists, files, etc) and only farm out to expect when needed. The last couple months I have employed this technique and my code has been drastically simplified.

Thanks for all the help Graham.

Posted by admin (Graham Ellis), 11 December 2005
My pleasure.  Many thanks for coming back to update us.

-- Graham

Posted by ShipConner (ShipConner), 19 March 2006
Ok maybe I have figured out how to say this

I would like to leverage Expect to overcome my bash scripts  interactive shortcommings. The idea is that Expect will spawn the script, the script executes its commands but when one of the commands results in a
username:
password:
i.e. ssh/scp

Expect should kick in and perform the interactive task and then allow the bash script to continue executing. Unfortunately it progresses this way istead:
) Expect starts script
) Bash script executes just untill the ssh command
) Expect sees the username/password prompts and logs in perfectly
) Expect never returns control back to the bash script allowing it to finish

Here is the Expect script that spawns the bash script:
set username [lindex $argv 0]
set password [lindex $argv 1]

set timeout -1
set prompt "(%|% |#|# |>|> |ftp>|ftp> |\\\$|\\\$ )$"
match_max 100000

spawn ./test.sh

#exp_internal 1

expect {
       timeout {puts "timed out"; exit 42}
       eof {puts "eof"; exit 99}
       -re "refused*" {puts "connection_refused $machine"; exit 86}
       -exact "continue connecting (yes/no)? " {send -- "yes\r"; exp_continue}
       -exact "assword:" {send -- "$password\r"; exp_continue}
       -re "erminal type*" {send -- "\r\r"; exp_continue}
       -re "$prompt" {sleep 2}
}
puts " "
--------------------------------------------------------

Here is the bash script spawned by Expect
#!/usr/local/bin/bash
uname -a
w
ssh -p 22 root@domain.com
uname -a
pwd
ps -aux
------------------------------------------------------

**Nothing after the ssh command is executed in the bash script. The Expect script continues sequently down its own script untill it is finished.

Is there a way I can make Expect give control back to bash? For example:
spawn bash_script
while != dead_bash_script
do
expect_this
done

Thank you as always Grahamaster

Posted by admin (Graham Ellis), 19 March 2006
I've got this sore throat and Lisa persuaded me to take some medication. "Won't send you to sleep" she said ... but it did and I've just woken up and I'm reading your post through bleary eyes  ...

I suspect the answer in interact.  Expect's interact command allows you to connect the spawned process direct to the keyoard and screen.  The big use of interact is to get you through complex login processes - perhaps if you have to go through several layers of firewall - then leave you running on the system you've reached ... but it may help in your case.

Other thoughts:

Does code within the bash script on the remote system do I/O from STDIN/STDOUT?   If so, you may wish to redirect the inputs especially from a file to that your expect process doesn't have to send the data

Can your bash script end by echoing a cardinal values - something that should never appear in the output stream - so that your waining expect process can pick that up and know to cleanly carry on?





Posted by ShipConner (ShipConner), 19 March 2006
>reading your post through bleary eyes  ...
Thank you, this forum and your company shows your dedication

>Interact
I always thought of interact was the same as Expect except that it could read/respond to pattern coming from the user(STDIN/STDOUT) as well as the spawned process.

But after some reading it seems Interact has some additional capabilities (page 353 Exploring Expect), namely the ability to redirect i/o for any program to any other program... not just the user. I will have to experiment to see if Interact can solve the problem.

>  Does code within the bash script on the remote system do I/O from >STDIN/STDOUT?

I aplogize, my setup is a little different than in my original posts from months before. Both the Expect script and bash script are on the same system. One of the lines in the bash script causes it to login to a remote system. Since Expect starts the bash script, the scripts I/O is linked directly to the Expect script. I know this becuase the Expect script reads/responds to the "username:" && "password:" prompts coming from the bash script.

>  Can your bash script end by echoing a cardinal values
I am not sure whay you mean...
If you mean can I have it echo something that expect would never see in a stream sort of like HEREDOC ... then yes. I am not understanding the direction since the bash script is sort of usurped as soon as it executes the ssh line.

Posted by admin (Graham Ellis), 19 March 2006
on 03/19/06 at 17:54:39, ShipConner wrote:
I aplogize, my setup is a little different than in my original posts from months before.


I wasn't aware that you had posted before.   This is posted from a new account opened just yesterday, so I had no history.

There has been nothing against the rules here in the same person having two accounts in the past - an oversight on my part as it's plainly a practise than confuses the user and shoud be discouraged.

Would you mind letting me know (by email to graham@wellho.net) what the second accound is called and which one you would prefer to keep, and I'll get them combined for you .... I'll then be able to get on and answer your question in the full knowledge of all previous correspondence on the subject

Edited to add - footnote - the previous "muxxum" account and the ShipConner account have been combined. I understand that a combination of an email address change and lost passowrd meant that the poste was no longer able to log in under his the previous account.

Posted by ShipConner (ShipConner), 19 March 2006
Here is what I Have tried so far:

1. Use exp_continue instead of sleep 2 in:
-re "$prompt" {sleep 2}

2. Wrapping my expect comment in a while loop
while 1 {
expect
same content as before
}

3. Interact. I am not even able to login using interact(seems like terminal problem but not sure). I also tried interat without the "-reset" as seen below(no luck)
-----------------------------------------
set username [lindex $argv 0]
set password [lindex $argv 1]

set timeout -1
set prompt "(%|% |#|# |>|> |ftp>|ftp> |\\\$|\\\$ )$"
match_max 100000

spawn ./test.sh

exp_internal 1
interact {
       -o -reset
       -re "refused*" {puts "connection_refused $machine"; exit 86}
       -exact "continue connecting (yes/no)? " {send -- "yes\r"; exp_continue}
       -exact "assword:" {send -- "$password\r"; exp_continue}
       -re "erminal type*" {send -- "\r\r"; exp_continue}
       -re "$prompt" {exp_continue}
}
puts " "
puts " "
---------------------------------------

It executes fine untill trying to send the password to remote system.
exp_internal 1 dies here:

spawn id exp5 sent <root@domain.com's password: >
root@domain.com's psend: sending "mypass\r" to { exp5 }
tty_set: raw = 0, echo = 1
command returned bad code: -101
   while executing
"interact {
       -o -reset
       -re "refused*" {puts "connection_refused $machine"; exit 86}
       -exact "continue connecting (yes/no)? " {send -- "y..."
   (file "./xxbash.tcl" line 16)
tty_set: raw = 3, echo = 0

**I am not sure if "code -101" is an error code from Expect, Bash, ssh. I am looking into it right now. So I can't even get to the point to where I can determine if Interact fixed my problem.

Posted by admin (Graham Ellis), 20 March 2006
Taking a step back, what are you expecting:

Code:
uname -a
w
ssh -p 22 root@domain.com
uname -a


to do?

As far as I can see, it attempts to log in to domain.com from the root account and whether on not it suceeds then repeats the uname -a on the machine on which the script is running - which is exactly what was done a couple of lines previously!

What it will actually do is log in to the domain (if the expect string is right) and hand there until either the expect times out, or until the remote machine closes the ssh though inactivity.  Perhaps you should be running nested expects (not something that thrills me!) or running each element of what looks like quite a simple bash script through expect and take out the bash level completely?

I fear that the latest tests are simply trying to fix what may be a design which has issues and won't lead to a simple solution - rather they're muddying the waters.


Posted by ShipConner (ShipConner), 20 March 2006
Graham,

My real objective is to create a wrapper around bash or ssh that will allow people who are not fluent in Expect to write scripts that usually aren't possible with bash. For example: it is impossible to do

#!/bin/bash

ssh -p me@me.com
password
do this
do that
exit

without first exchanging pub/private keys. Exchanging keys requires a significant deployment in enterprise level organizations with thousands of machines (I have worked for serveral and this is always a problem, not to mention revisting the process for each new employee).

To be honest I have already solved this with an Expect only script. I am trying to empower BASHERS by creating a wrapper that will not require them to know any Expect.. So where the previos example is impossible this would be:

Xssh -p me@me.com
password
do this
do that

The commands I used are just examples. Those commands I used (uname -a)don't pertain to a specific problem.  I am just using them as filler commands.

I apologize I can see I am wasting your time. I will let you know if I ever solve it.

Posted by admin (Graham Ellis), 21 March 2006
I do think that this is one that would be best solved / looked at by an hour or two's headbashing in the same room and isn't working too well via the forum.  And I know that's not practical as you're half a world away.   Glad that it's not a business-critical crisis that you're trying to solve   anyway - yes, please do keep us posted!

Graham



This page is a thread posted to the opentalk forum at www.opentalk.org.uk and archived here for reference. To jump to the archive index please follow this link.

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2022: Well House Manor • 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • FAX: 01144 1225 793803 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho