Hi there,

Documenting a method of compiling a kernel for a slow machine
on a faster machine, and installing kernel + modules over NFS
link.

Why? Old laptops with small hard drives, little room for compiler
+ development tools, and way too slow for compiling custom kernels.

Requires: local network with a faster compile host, NFS running,
and target machine needs to export its root directory to localnet.

An installkernel script is used to automate some of the install,
see: <http://bugsplatter.mine.nu/bash/kernel/installkernel.gz>.
While not required, it is what I've been using for a long time.
The installkernel script is placed in /root/bin and is called
during the linux-kernel 'make install' script execution.


target requirements
````````````````````
NFS export the / partition, if you have a separate /boot, export
that too:

~$ cat /etc/exports
# See exports(5) for a description.
# This file contains a list of all directories exported to other computers.
# It is used by rpc.nfsd and rpc.mountd.

# for hosted kernel compile
/ 192.168.1.0/24(sync,rw,no_root_squash)


host
`````
define target's mountpoints:

~$ cat /etc/fstab
# /etc/fstab for slackware on sempro -- 2006-04-02
....
# we do hosted compiles for these slow boxen:
deltree:/ /home/deltree nfs noauto,user,hard,intr
hal:/ /home/hal/ nfs noauto,user,hard,intr

I keep kernel source under ~/linux and the various source trees are
hardlinked to save space while keeping current source state:

~/linux$ ls -1
host-deltree/
host-hal/


custom kernel example
``````````````````````
For this, I compile 2.4.33-pre3 for the laptop called 'hal' using
the .config from previously compiled 2.4.32-hf32.4 kernel:

$ cd host-hal
$ cp -al ../linux-2.4.33-pre3 .
$ cd linux-2.4.33-pre3
$ make mrproper
$ mount /home/hal/
$ cp /home/hal/boot/config-2.4.32-hf32.4 .config
$ make oldconfig; make dep; time make bzImage modules
....
real 3m14.053s
user 2m58.540s
sys 0m13.300s

Install kernel and modules to target over NFS:

$ su
# INSTALL_PATH=/home/hal/boot make install
# INSTALL_MOD_PATH=/home/hal make modules_install
# exit
$ umount /home/hal/

Open a root console to the target, verify expected kernel was
installed, edit /etc/lilo.conf if required, then run lilo:

root@hal:~# ls -lrt /boot/
....
-rw-r--r-- 1 root root 17314 2006-05-23 11:42 config-2.4.33-pre3
-rw-r--r-- 1 root root 795946 2006-05-23 11:42 bzImage-2.4.33-pre3
-rw-r--r-- 1 root root 300790 2006-05-23 11:42 System.map-2.4.33-pre3

root@hal:~# lilo
Added 2.4.33-pre3 *
Added 2.4.32-hf32.4
Added slack <<== always keep the distro kernel around
Added windoze

Timesaver? You betcha, box 'deltree' takes about 80 minutes to
compile a 2.4 kernel on a pentium-mmx/233. I've not compiled a
kernel on the 'hal' laptop, p100 with 24MB memory? Take hours.

For 2.6 series kernels merge the install commands like this:
# INSTALL_PATH=/home/hal/boot INSTALL_MOD_PATH=/home/hal make install

Grant.
--
Memory fault -- brain fried

Grant wrote:

> Requires: local network with a faster compile host, NFS running,
> and target machine needs to export its root directory to localnet.

Or a staging directory on the compile host and scp to copy the files
over (or even "tar |ssh ... tar"), which is likely a much better idea
than exporting the root directory, such that the kernel and modules can
be overwritten via NFS ... did this not raise any red flags in your
mind at all???

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

On Tue, 23 May 2006 03:56:02 +0000 (UTC), Sylvain Robitaille <syl@alcor.concordia.ca> wrote:

>Grant wrote:
>
>> Requires: local network with a faster compile host, NFS running,
>> and target machine needs to export its root directory to localnet.
>
>Or a staging directory on the compile host and scp to copy the files
>over (or even "tar |ssh ... tar"), which is likely a much better idea
>than exporting the root directory, such that the kernel and modules can
>be overwritten via NFS ... did this not raise any red flags in your
>mind at all???

Yeah, but the only hostile user here is me in a bad mood

Old kernel build script used to make a tarball for host compiles,
but I changed to NFS mount a couple years ago to get easy access
to the target's /boot/config-* files. 2.6 series build has a
tarball option, but not 2.4, and me be running 2.4 series for a
while yet. I use same script for either series kernel build.

Hmmm, next time I work on the installkernel script I'll see about
scp or perhaps rsync. tar | ssh ... tar seems a tad hairy, it's
tar's man page what scares me off using tar beyond 'cvzf' and
friends A full screen of parameters summary is a bit of an
insult, no?

I'd like to have the script modify lilo.conf too, tweak minor
versions, particularly now the frequency of 2.6-stable patches
has rocketed. Already started on a delta-build script that backs
out old version and patches to new version.

Grant.
--
Memory fault -- brain fried

On Tue, 23 May 2006 03:56:02 +0000 (UTC), Sylvain Robitaille <syl@alcor.concordia.ca> wrote:

>Grant wrote:
>
>> Requires: local network with a faster compile host, NFS running,
>> and target machine needs to export its root directory to localnet.
>
>Or a staging directory on the compile host

As I pressed send... I already have a common shared resource
exported from the firewall box, all linux boxen mount /home/share
rw for kernel source + patches, and to write out stuff for transfer.

Grant.
--
Memory fault -- brain fried

Grant wrote:

>> did this not raise any red flags in your mind at all???
>
> Yeah, but the only hostile user here is me in a bad mood

Are you willing to declare that your home network is completely
impermeable?

> tar | ssh ... tar seems a tad hairy, ...

It isn't, once you get used to it. I use that approach practically
daily.

> tar's man page what scares me off using tar beyond 'cvzf' and
> friends A full screen of parameters summary is a bit of an
> insult, no?

You understand "tar cf - -C /some/path .", right? And you understand
"(cd /new/path && tar xpvf -)", right? Now put them together, with a
pipe to ssh in between:

tar cf - -C /some/path . |ssh somehost "(cd /new/path && tar xpvf -)"

There are even more possibilities, given suitable options to tar, but
this is the way I use this most often.

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In alt.os.linux.slackware, Sylvain Robitaille dared to utter,
> tar cf - -C /some/path . |ssh somehost "(cd /new/path && tar xpvf -)"
>
> There are even more possibilities, given suitable options to tar, but
> this is the way I use this most often.

I prefer to use netcat myself, often for moving large amounts of files
very quickly over a network. I typically setup two screens.

Host screen
```````````
$ tar cv some_path | nc server port

Server screen
`````````````
$ nc -w 60 -l -p port | tar xv

Then I just disconnect the screens and go my merry way. This is a
great way to transfer a dozen gigabytes of small files (say when
performing a full backup of a windows machine before a format) because
of how quickly it moves the data. As long as your disk speed is higher
than your bandwidth, your network will be saturated with very little
overhead.

Sylvain's example is superior for moving reletively small amounts of
data because you do not have to login to the remote system, or worry
about setting up a netcat service. However, the use of ssh adds a lot
of encryption overhead in both CPU time and packet size, making this a
less efficient way for moving data on the order of several hundred
megabytes and higher across a LAN.

- --
It is better to hear the rebuke of the wise,
Than for a man to hear the song of fools.
Ecclesiastes 7:5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.7 (GNU/Linux)

iD8DBQFEcyuSz8zcalmVmBkRAr6xAKCTvDLxX/FB1RG+OCoBtVkBSu+rUgCgq0Nm
Z+s7oLmliNPb2F25chl7umI=
=4VZI
-----END PGP SIGNATURE-----

+Alan Hicks+ wrote:

> Sylvain's example is superior for moving reletively small amounts of
> data because you do not have to login to the remote system, or worry
> about setting up a netcat service. However, the use of ssh adds a lot
> of encryption overhead in both CPU time and packet size, making this a
> less efficient way for moving data on the order of several hundred
> megabytes and higher across a LAN.

That hasn't been a problem, in practice, either on my relatively slow
home network, or on 100Mb switched network at work. My home systems
even include some old 486s with relatively small amounts of memory.
What I've shown is what I use to move large amounts of data, even across
wireless (802.11b) and DSL links. The encryption overhead, in my opinion,
is just a matter of course. If I want to move "relatively small amounts
of data", I just use scp.

I wouldn't want to move gigabytes of a system's backup data (Windows or
otherwise), over an unencrypted link created with netcat, not even on
my quite well protected home network, unless the system in question
contained exactly 0 bytes of sensitive information. I've been using Ssh
for so many years, both for interactive access, and for tunnelling
traffic and pipes that I don't even notice the overhead.

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

On Tue, 23 May 2006 15:15:46 +0000 (UTC), Sylvain Robitaille <syl@alcor.concordia.ca> wrote:

>Are you willing to declare that your home network is completely
>impermeable?

The localnet IP addresses (*.mire.mine.nu) resolve to the firewall's
public IP from the world side, and:

~# iptables-save |grep FORWARD
:FORWARD DROP [3:144]
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.1.0/255.255.255.0 -d 192.168.2.0/255.255.255.0 -i eth0 -o eth1 -j ACCEPT
-A FORWARD -s 192.168.2.0/255.255.255.0 -d 192.168.1.0/255.255.255.0 -i eth1 -o eth0 -j ACCEPT
-A FORWARD -s 192.168.1.0/255.255.255.0 -i eth0 -o ppp0 -j egress
-A FORWARD -s 192.168.2.0/255.255.255.0 -i eth1 -o ppp0 -j egress
-A FORWARD -j LOG --log-prefix "ForDrop policy " --log-level 6

See <http://bugsplatter.mine.nu/junkview/iptables-save-masked.gz> for
the rest. There are no localnet ingress rules, and no public login
services offered on the firewall.

>You understand "tar cf - -C /some/path .", right? And you understand
>"(cd /new/path && tar xpvf -)", right? Now put them together, with a
>pipe to ssh in between:
>
> tar cf - -C /some/path . |ssh somehost "(cd /new/path && tar xpvf -)"

Given an example, I can work through it (with man tar) and expand on
it (or avoid it One thing I notice is that a p100 take quite some
time (a few seconds) to startup a ssh session -- the number crunching
for the key exchange, I suspect.

Do you run an ssh agent for RSA authentication?

Grant.
--
WinXP: Access Start->Turn Off Computer, then while holding Ctrl-Alt-Shift,
left click on Cancel. This terminates Windows Explorer...

Grant wrote:

>> Are you willing to declare that your home network is completely
>> impermeable?
>
> The localnet IP addresses (*.mire.mine.nu) resolve to the firewall's
> public IP from the world side, and: ...

I have a similar setup at home, and there are still very few unencrypted
protocols that are used on that network (LPD and SMTP come to mind as
two examples; DNS, SMTP, and HTTP are port-forwarded from the gateway
system to their respective servers, and all hosts involved are quite
well protected). I still would not declare my home network completely
impermeable. There are too many components whose underlying mechanisms
I do not have a thorough understanding of, though I feel I've done as
well as can be to protect my systems.

Then again, I work in an environment where the network is considered
hostile, so perhaps I'm more paranoid than you are. On the other hand,
that work environment gives me plenty of experience protecting hosts
under constant attack. My home network isn't under constant attack, and
I use the same measures to protect my home systems, and I still would
not consider them completely impermeable.

> One thing I notice is that a p100 take quite some
> time (a few seconds) to startup a ssh session ...

Shrug ... a few seconds isn't going to make a huge difference in the
long run, is it? I do find that OpenSSH is slower to startup than the
original Ssh, though (when compared on the same system). What you're
seeing may be caused, at least in part, by that.

> Do you run an ssh agent for RSA authentication?

Only from my workstation, so for the purposes of this discussion, "not
usually".

--
----------------------------------------------------------------------
Sylvain Robitaille syl@alcor.concordia.ca

Systems and Network analyst Concordia University
Instructional & Information Technology Montreal, Quebec, Canada
----------------------------------------------------------------------

On Tue, 23 May 2006 11:33:24 -0500, +Alan Hicks+ <alan@lizella.netWORK> wrote:

>In alt.os.linux.slackware, Sylvain Robitaille dared to utter,
>> tar cf - -C /some/path . |ssh somehost "(cd /new/path && tar xpvf -)"
....
>I prefer to use netcat myself, often for moving large amounts of files
>very quickly over a network. I typically setup two screens.
>
>Host screen
>```````````
>$ tar cv some_path | nc server port
>
>Server screen
>`````````````
>$ nc -w 60 -l -p port | tar xv

Never used netcat, I'd use rsync for casual large transfers, or
the usual (to me) cp -a ... over NFS. Thing is, there are so many
tools to choose from, and I find myself keeping to a small set of
tools I trust, even though they're are perhaps better ways to do
a task.

Sylvain's focus on security is the future we all need to consider
as linux becomes a popular target for takeovers --> such as the
naive LAMP boxen people put up in server or coloc. farms that get
rooted and used to flood yet more crap onto the 'net.

Grant.
--
WinXP: Access Start->Turn Off Computer, then while holding Ctrl-Alt-Shift,
left click on Cancel. This terminates Windows Explorer...