Sunday, February 10, 2008

Getting your parents to run Fedora: Challenge 1

Problem 1: Remote system administration
As other people in the session shared their similar philanthropic tech support stories we soon discovered our first problem. A problem which has led to much headache for all involved and marred the name of open source. The issue we speak of in this entry is that of remote system administration.

You can't be serious
It definitely seems odd that a group of people consisting of either developers or system administrators would face challenges with remote administration. I mean, just about every machine I work on is "remote". It's typically a Xen guest whose physical host location is both unknown and unimportant to me. But the fact of the matter is that, when dealing with our family, we're often thrown into a world (almost) completely unfamiliar to us; the world of people who do not attend FUDcons.

"Oh, but I am serious" or Phone support for the inlaws
To make things a little more concrete I'll share a little about a strange tradition in our household. Each year my wife and I fly out to visit her relatives. With impeccable regularity, during our vacation one of their Windows machines goes up in flames due to some mixture of bugs and malware and, with similar predictability, I install Fedora on their machine. The sad part to the story is that every year they end up having to remove Fedora and switch back to Windows simply because we fly back home and they don't have anyone else nearby that can help them. The truth is they have equally as many problems with Windows. It just happens to be the case that they have had years of experience with 'workarounds'. I can't tell you how many times we've had conversations on the phone like this:
mother-in-law: The internet is broken on my laptop
me: Opps, that was probably my fault. Remember that thing I mentioned to about Ogg Vorbis? Well when I ssh'd into your machine this morning to make it so that you could view our family videos I accidentally upgraded your kern...

*mother-in-law* interupts,

mother-in-law: Uhh... what can I type to fix this? I need to check my email.
me: Press alteff too. Then type 'nome terminal'. Then type 'pseudo sue dash enter'. Then type 'yum update eye dubya...'

(I'm trying my best to faithfully represent the role poor mobile phone connectivity plays in to these scenarios.)
Current recommendation for my inlaws:
I was quite surprised when several people from the Fedora project actually suggested not installing Fedora in this particular case. After talking a bit more what they really meant was, "don't install Fedora, yet". In certain cases it can be almost traumatic to have their entire digital world flipped on them. Find _good_ open source solutions to the areas where they are locked in. Once successful the switch to Fedora will be a lot smoother. After all, it's typically the case that a person only uses a handful of rather basic apps and they only have one or two apps where they are truely 'locked in'. In my mother-in-law's case it's Sibelious. Sadly, GNU Denemo didn't work well for her.

For remote support we really need something better than
  1. explain how to activate desktop sharing
  2. reverse an ssh connection
  3. port forward to their vino server
  4. launch vnc viewer over lousy internet connection and try to couch family member on what button to click on to fix their problem
The above steps usually take 20-30 minutes for me to setup and, nine times out of ten, the fix involves something as simple as, "oh, you forgot to type in the browser. That's why your email isn't working".

Wish list for remote tech support
  • An effective way to securely establish connections with nontechnical users and their trusted, penguin-loving friends. When connecting two people that are both on networks with some form of NAT you are frequently forced to use a third, internet addressable, machine in order to establish a connection. Obviously you can get around the use of a third party if you control the router on your end, but that's not always the case. Say my mom calls from home and I'm at coffee shop or at work.
  • A disaster recovery strategy that protects our family members from being forced to return closed source alternatives. With my inlaws some unrecoverable (at least over the phone) situation always comes up and they are forced to do the only thing they know how to--pay someone to install Windows.
Possible solutions
  • Use a service to spin up secured Fedora virtual machines over the internet. From there have an applet on their dock to request a connection from their trusted friend. Some form of remote desktop connection is then made over a tunnel ssh connection that is brokered by the VM. Of course using such a service would always require money. In the case of the EC2 it's pretty cheap in comparison to the cost of paying some Geek Squader to fix things. With Amazon's DevPay (A service that allows people to pay for usage of VMs) it would be possible to even have a system for Fedora community members to raise support for the Fedora project by volunteering to fix people's machines.
  • Koan/Cobbler to allow remote provisioning. Profiles could be created for sundry machine configurations that would lay down a working machine.
  • Customized LiveCDs or recovery DVDs. This shouldn't be too hard with the Fedora tooling and fits quite well with Koan/Cobbler.
  • Another interesting use case for a live CD would be to have an automated method to back up their data before wiping everything and starting over. In the post installation their data could be restored.
Up Next
The second challenge: Usability


David said...

What I do over a slower speed internet connetion, YRMV.

On the users computer. Create a separate user account ( eg "sysop" ) for you to remotely log into via ssh.

Insure that the internet connection ( modem/router/firewall ) listens and forwards ssh correctly, and that sshd allows local port forwarding. If the user does not have a static IP address then setup a dynamic DNS service ( ie ) . Also make sure that remote desktop is turned on with either an assenting click or password.

Remotely log into the box:
"ssh -C -R 55000:localhost:5500".

Install vncserver and vnc .
As sysop create a temporary vnc X server to use a viewport for a slow or unreliable internet connection:
"vncserver -geometry 800x600 :16"
( it will prompt for a password the first time and by default use the twm window manager and start a xterm ). You can adjust the geometry to the internet connection.

On your computer, fire up new terminal window, insure vnc is installed ( vinagre doesn't support -listen option yet ) have it listen to port 5500.
"vncviewer -listen".

Then via the ssh terminal
"vncconfig -display :16 -connect localhost:55000"
A window to the remote vnc server will then appear on your computer. In that window in the xterm provided type.
"vncviewer localhost". You will now have a vnc viewport onto the users desktop.

This can work really well and does not require granting the support person root access. If the internet connection disconnects then you just re ssh into the box and kill the old ssh connection.

Afterwards kill the vnc X server with:
"vncserver -kill :16".

Also ( although i have only done this once ), you can use
connected to the vnc X server to record exactly what you are doing and embed it in a webpage for the hapless user to refer back to.

It would not be to difficult to bash script all this and even package it in a rpm.

Come to think of it a Fedora project hosted dynamic DNS service might be a good idea.

What is really needed is a community based remote support service, which I shall expand upon in a future blog entry.

brenton said...

David, thanks for the tips. It's definitely time we address this issue formally and perhaps package a solution that we can all contribute to.

I really like the idea of forming a community around remote support. Herlo created which would might be a good place for ideas to percolate.

pmacedo said...

One important thing to be done IMHO is improve the built-in servers in Gnome and KDE.

Last time I tried to use KDE's built-in session sharing, it was awfully slow, even considering that the host was on 100mbit lan and the client was on the same LAN over wireless using 802.11g . Put that over a slow internet connection and you have a nightmare in your hands.

Kevin Fenzi said...

As far as remote access, how about openvpn? It's easy, secure and nice.

So on your machine you setup dyndns or some other way to keep a dns->ip mapping. Then on the machines you want to support just install openvpn and have it connect to your dns name.

You can then use the vpn to login and do whatever admin work you need to do (as long as the underlying network is up). Openvpn will take care of reconnecting and encrypting.


my google reader feed