If you use vip instead of ip address, in your tnsname.ora file, you can avoid this problem.
you have two cases in here.
1) shutdown instance, but listener still up in server1
when client connections connect to listener in server1, listener in server 1 will relocate those connections to instance in server2.
Check remote_listener in init parameter file
2) shutdown intances and listener in server1, but vip still up.
vip will inform the incoming connection to connect to the second ip address in tnsname.ora file.
You can try to use outbound_connect_timeout in sqlnet.ora file in client side.
