[Mono/Windows/Linux/C#] C# Mono No STDOUT or STDERR To Console on Windows

December 19, 2011 | Posted by Chuck Kozler

Hey Guys,

This is just a quick little post.

I found that on Linux, regardless of the term or shell being used, I would still have all output, both stderr and stdout, being written to my shell when I would load up my app from the command line.

However, on Windows, I was not able to see any of my output- even exceptions! This is a bit ridiculous in my opinion but from what I can tell, Microsoft has made it this way and it seems to be because windows applications do not connect to a console. So, in this instance, we want cmd.exe to be our parent process and inevitably attach to that as our console.

If you want to return the text to your cmd.exe, thus giving you access back to Console.WriteLine and exception messages, then here is a quick little code snippet you can use.

Note I implemented this through monodevelops project wizard for deploying a GTK# application which creates all base code for writing a GTK# application- this creates Main.cs and MainWindow.cs amongst other reference files (like the icon, etc). That being said, this code will look different and you may not have the same issues I had

How to recreate the symptoms I had:

1.) Install Mono and MonoDevelop on Linux
2.) Create a new project in MonoDevelop for a GTK# with a target of 2.12
3.) Use any Console.WriteLine( “…” ) function call anywhere
4.) See it outputs as expected on Linux
5.) Install Mono on Windows 7
6.) Copy your app to Windows computer
7.) Execute cmd.exe
8.) Execute C:\path\to\your\mono\installation\bin\mono.exe C:\path\to\your\app.exe
9.) Notice you have no output!

Applying the code below will fix this

Finally, using this will stop you from being able to pipe the output to a text file. I have not yet found a fix for this but will keep you posted – I am guessing it could be a lot of ninja-ing and hackery with GetStdHandle and the likes but it really doesnt matter to me at this point in time since my logging is done both to console and to a file via my app directly to Streams.

Main.cs

using System;
using Gtk;
using System.Runtime.InteropServices;
 
 
namespace Test
{
 
	static class MainClass
	{
		//Import kernel32.dll for implementing AttachConsole
		[DllImport( "kernel32.dll" )]
		static extern bool AttachConsole( int dwProcessId );
                private const int ATTACH_PARENT_PROCESS = -1;
 
		public static void Main (string[] args)
		{
			// Detect that we are _NOT_ running on Linux because this call 
			// will obviously fail on Linux and throw an exception
			int p = (int) Environment.OSVersion.Platform;
            	        if ( ! ((p == 4) || (p == 6) || (p == 128)) ) {
 
				// Then attach to the parent process instead of the console
				// which is now cmd.exe
				AttachConsole( ATTACH_PARENT_PROCESS );
			}
 
			Console.WriteLine( "Calling init" );
			System.Diagnostics.Debug.WriteLine( "Debug line" );
			Application.Init ();
 
                        //See base code below
			MainWindow win = new MainWindow ();
			Console.WriteLine ( "Calling show" );
			win.Show ();
			Application.Run ();
		}
	}
}

MainWindow.cs

using System;
using Gtk;
 
public partial class MainWindow: Gtk.Window
{	
	public MainWindow (): base (Gtk.WindowType.Toplevel)
	{
		Console.WriteLine( "MainWindow constructor called" );
		Build ();
	}
 
	protected void OnDeleteEvent (object sender, DeleteEventArgs a)
	{
		Console.WriteLine ( "Quitting" );
		Application.Quit ();
		a.RetVal = true;
	}
}

Sphere: Related Content

Category Categories: Computers, Linux/Unix, Programming, Ubuntu, Windows | Tag Tags: can not see stdout linux mono windows, console redirection windows linux c#, mono no stdout from gtk#, no console output from mono application, no console output to cmd.exe windows mono linux c#, no console text mono windows linux, no stderr mono windows linux, no stdout gtk#, windows linux, windows linux C#, windows linux C# no stdout stderr | 2 Comments

[Linux/Perl] Generic Cron Backup Script v2

October 14, 2011 | Posted by Chuck Kozler

This is the diff from whats been updated

  line diff
     1.1 --- a/scripts/backup_scripts/doCronBackup.pl	Fri Oct 14 15:17:03 2011 -0400
     1.2 +++ b/scripts/backup_scripts/doCronBackup.pl	Fri Oct 14 21:11:49 2011 -0400
     1.3 @@ -191,8 +191,14 @@
     1.4  	elsif( $g_useRdiffBackup ) {
     1.5  		
     1.6  		&print( "* We are using rdiff-backup" );
     1.7 +		
     1.8 +		# Lets first find on the directory we are backing up and create the directories on the remote server
     1.9 +		my $createRemoteDirsCommand = "find " . $g_cmdOptions{ 'local_dir'} . " -type d | xargs -I{} ssh " . $g_cmdOptions{ 'host' } . " \"mkdir -vp " . $g_cmdOptions{ 'remote_dir' } . "{} \"";
    1.10 +		&print( "Create remote directories: " . $createRemoteDirsCommand );
    1.11 +		&exec( $createRemoteDirsCommand );
    1.12 +		
    1.13  		my $rdiffDefaultArgs = "--terminal-verbosity=7 --force";
    1.14 -		my $rdiffCommand = "${g_backupBin} ${rdiffDefaultArgs} " . $g_cmdOptions{ 'extra_opts' } . "  " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'host'} . "::" . $g_cmdOptions{ 'remote_dir' };
    1.15 +		my $rdiffCommand = "${g_backupBin} ${rdiffDefaultArgs} " . $g_cmdOptions{ 'extra_opts' } . "  " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'host'} . "::" . $g_cmdOptions{ 'remote_dir' } . $g_cmdOptions{ 'local_dir' };
    1.16  		&print( "Command to run: " . $rdiffCommand );
    1.17  		my $rdiffRetval = &exec( $rdiffCommand );
    1.18  		if( $rdiffRetval != 0 ) {

and the completed script

#!/usr/bin/perl
 
# You'll see why I did this later on (in case I change the package name I can easily change it back)
# Removed: This doesnt seem to work as expected on all versions of perl
#use constant PACKAGE => doCronBackup;
 
#package PACKAGE;
 
use strict;
use warnings;
 
use Getopt::Long;
 
# Global variable for the command line
# options available to the script
my %g_cmdOptions = (
			'method' 		=> "rsync",					# Method = rsync or rdiff-backup. Defaults to rsync since that is standard on systems
			'host'	 		=> undef,					# The host as defined in the ssh_config file
			'local_dir' 		=> undef,					# The local directory we are backing up
			'remote_dir'	 	=> undef,					# The directory we are putting it to 
			'success_email' 	=> "prod_systems\@fixflyer.com",		# The email to send success emails to 
			'failure_email' 	=> "sysops\@fixflyer.com",			# The email to send failure emails to
			'extra_opts'		=> undef,					# Extra options we want to pass to rsync or rdiff-backup (this is not checked)
		);			
 
 
# This will be later populated by init when we obtain the local_dir value and method value
# eg: /etc/httpd with rsync will give us rsync.etc.httpd.log in /tmp
my $g_logFile = undef;
my $g_hostname = ""; 
 
my $g_useRsync = 0;
my $g_useRdiffBackup = 0;
 
# Depending on the value specified in method command line option, this will be either the abs path to rsync or rdiff-backup
my $g_backupBin = undef;
 
sub getLogTime {
	my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime( time );
	my $date_time = sprintf( "%4d%02d%02d-%02d%02d.%02d",$year+1900,$mon+1,$mday,$hour,$min,$sec );
	return $date_time;
}
 
 
sub print( ) {
 
	print "[ " . &getLogTime() . " ] " . $_[0] . "\n";
 
	if( defined $g_logFile ) {
		if( !open( FILE," >>$g_logFile" ) ) {
			warn( "! Can not open() log file ${g_logFile}: at line " . __LINE__ . "\n" );
		}
 
		CORE::print( FILE "[ " . &getLogTime() . " ] " .$_[0] . "\n" );
		close( FILE );	
	}
}
 
sub exec( ) {
	my $strCommand = $_[0];
	use IPC::Open3;
	use Symbol qw( gensym );	
	use IO::File;
	local *OUT = IO::File->new_tmpfile;
	local *ERR = IO::File->new_tmpfile;
	my $iPID = open3( gensym, ">&OUT", ">&ERR", "$strCommand" );
	waitpid( $iPID, 0 );
	$? = $? >> 8;
	my $iRetval = $?;
	seek $_, 0, 0 for \*OUT, \*ERR;
	while( <ERR> ) {
		chomp( $_ );
		my $txt = "err: " .  $_;
		&print( "$txt" );
	}
	while( <OUT> ) { 
		chomp( $_ );
		my $txt = "out: " . $_;
		&print( "$txt" );
	}
 
	return $iRetval;
}
# doInit(): Used to do the initialization of the script. Get options, do basic checks on globals and command options
sub init( ) {
	GetOptions(
	       		"method=s" 		=> 	\$g_cmdOptions{ 'method' },
			"host=s"		=>	\$g_cmdOptions{ 'host' },
			"local_dir=s"		=>	\$g_cmdOptions{ 'local_dir' },
			"remote_dir=s"		=>	\$g_cmdOptions{ 'remote_dir' },
			"success_email=s" 	=>	\$g_cmdOptions{ 'success_email' },
			"failure_email=s"	=>	\$g_cmdOptions{ 'failure_email' },
			"extra_opts=s"		=>	\$g_cmdOptions{ 'extra_opts' }
	);
 
	# Check for empty values
	while( ( my $key, my $value ) = each( %g_cmdOptions ) ) {
		# We dont care about the extra_opts value. The utility (rsync or rdiff-backup) will fail if the user supplies a bad one
		if( $key eq "extra_opts" ) {
			if( defined $g_cmdOptions{'extra_opts'} ) {
				next;
			}
			else {
				$g_cmdOptions{'extra_opts'} = "";
			}
			next;
		}
 
		if( not defined $value ) {
			&print( "! Command line option $key is empty. Please supply a value" );
			exit 5;
		}	
 
	}
 
	# Construct the log file name and delete it if it exists
	my $logFileBaseDir = "/tmp";
	my $logFileName = $g_cmdOptions{ 'method' } . $g_cmdOptions{ 'local_dir'};
	$logFileName =~ tr/\//./;
	$g_logFile = $logFileBaseDir . "/" . $logFileName . ".log";	
	&exec( "rm -v " . $g_logFile ) if -f $g_logFile;
	&print( " + Log file name \t\t=>\t\t $g_logFile" );
 
 
	$g_hostname = `hostname`;
	chomp( $g_hostname );
 
	# Show the values for what we have been supplied
	while( ( my $key, my $value ) = each( %g_cmdOptions ) ) {
	       	# I hate my OCD	
		if ( $key eq "method" or $key eq "host" ) {
			&print( " + $key \t\t\t=>\t\t $value " );
		}
		else {
			$value = "- none specified -" if not defined $value;
			&print( " + $key \t\t=>\t\t $value " );
		}
	}
 
	# Check that we have our backup bin (rsync or rdiff-backup) available
	if( $g_cmdOptions{ 'method' } eq "rsync" ) {
		$g_backupBin = `sh -c "command -v rsync"`;
		chomp( $g_backupBin );
		if( not defined $g_backupBin or $g_backupBin eq "" ) {
			&print( "Can not get rsync bin. Please check your PATH and that the command exists" );
			exit 255;
		}
		&print( " + Backup command \t\t=>\t\t $g_backupBin" );
		$g_useRsync = 1;
	}
	elsif( $g_cmdOptions{ 'method' } eq "rdiff-backup" ) {
		$g_backupBin = `sh -c "command -v rdiff-backup"`;
		chomp( $g_backupBin );
		if( not defined $g_backupBin or $g_backupBin eq "" ) { 
			&print( "Can not get rdiff-backup bin. Please check your PATH and that the command exists" );
			exit 255;
		}
		&print( " + Backup command \t\t=>\t\t $g_backupBin" );
		$g_useRdiffBackup = 1;
	}
	else {
		&print( "! Invalid backup method supplied.  Valid values are 'rsync' or 'rdiff-backup' (without single quotes). Was supplied " . $g_cmdOptions{ 'method' } );
	}
 
	&print( " + Hostname \t\t=>\t\t $g_hostname" );
	&print( " + Script \t\t\t=>\t\t " . __FILE__ );
}
 
sub execute( ) {
	if( $g_useRsync ) { 
 
		&print( "* We are using rsync" );
		my $rsyncDefaultArgs = "-Rakv";
		my $rsyncCommand = "${g_backupBin} ${rsyncDefaultArgs} " . $g_cmdOptions{ 'extra_opts' } . " " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'host' }.":".$g_cmdOptions{ 'remote_dir' };
		&print( "Command to run: " . $rsyncCommand );
		my $rsyncRetval = &exec( $rsyncCommand );
		if( $rsyncRetval != 0 ) {
			&print( "! Errors running rsync" );
			my $emailCommand = "cat $g_logFile | mail -s \"ERRORS Encountered: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'failure_email'} . "";
			&print( "+ Sending email: " . $emailCommand );
			&exec( $emailCommand );
 
		} else {
			&print( "+ Succesfully executed rsync" );
			my $emailCommand = "cat $g_logFile | mail -s \"Success: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'success_email'} . "";
			&print( "+ Sending email: " . $emailCommand );
			&exec( $emailCommand );
		}
 
 
	}
	elsif( $g_useRdiffBackup ) {
 
		&print( "* We are using rdiff-backup" );
 
		# Lets first find on the directory we are backing up and create the directories on the remote server
		my $createRemoteDirsCommand = "find " . $g_cmdOptions{ 'local_dir'} . " -type d | xargs -I{} ssh " . $g_cmdOptions{ 'host' } . " \"mkdir -vp " . $g_cmdOptions{ 'remote_dir' } . "{} \"";
		&print( "Create remote directories: " . $createRemoteDirsCommand );
		&exec( $createRemoteDirsCommand );
 
		my $rdiffDefaultArgs = "--terminal-verbosity=7 --force";
		my $rdiffCommand = "${g_backupBin} ${rdiffDefaultArgs} " . $g_cmdOptions{ 'extra_opts' } . "  " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'host'} . "::" . $g_cmdOptions{ 'remote_dir' } . $g_cmdOptions{ 'local_dir' };
		&print( "Command to run: " . $rdiffCommand );
		my $rdiffRetval = &exec( $rdiffCommand );
		if( $rdiffRetval != 0 ) {
			&print( "! Error running rdiff-backup. Please view the logs - return code was: $rdiffRetval" );
			# Send email to error list
			my $emailCommand = "cat $g_logFile | mail -s \"ERRORS Encountered: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'failure_email'} . "";
			&print( "+ Sending email: " . $emailCommand );
			&exec( $emailCommand );
 
		} else {
			&print( "+ Successfully executed rdiff-backup" );
			my $emailCommand = "cat $g_logFile | mail -s \"Success: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'success_email'} . "";
			&print( "+ Sending email: " . $emailCommand );
			&exec( $emailCommand );
		}
 
 
		# And exit with the return value of rdiff-backup
		exit $rdiffRetval;
 
	} else {
		&print( "You shouldn't be here, silly" );
		exit 155;
	}
 
}	
 
&init( );
&execute( );

Category Categories: Computers, Linux/Unix, Networks/Servers, Programming, Ubuntu | No Comments

[Linux/Perl] Generic Cron Backup Script

October 14, 2011 | Posted by Chuck Kozler

This is a pretty basic cron script for generic backups with rdiff-backup and rsync.

TODO in the next version is to auto create the directory tree- something like

find $g_cmdOptions{'local_dir'} -type d | xargs -I{} ssh $g_cmdOptions{'host'} "mkdir -pv $g_cmdOptions{'remote_dir'}/{}"

Right before we do the actual transfer. That would get the entire list of directories and then auto create them causing us to avoid the “can not create directory” error with rdiff backup (I think rsync has this built in somewhere)

For now, here is a how to use it:

1. The argument in –host needs to be visible from the ssh_config file and would most likely be used with SSH keys

Example:

$ perl doCronBackup.pl --local_dir /etc/httpd/ --remote_dir /store/monitor.ny1.prod/backup/etc/httpd/ --host backup --method rdiff-backup
$ perl doCronBackup.pl --local_dir /etc/nagios/ --remote_dir /store/monitor.ny1.prod/backup/etc/nagios/ --host backup --method rdiff-backup
# Now because rdiff-backup doesn't have an override for ignoring when a file has been updated and terminates (its to stop file corruption but we should have a switch to work with!) I have
# incorporated an rsync feature as well for the instance you need to rsync log files that consistently change
$ perl doCronBackup.pl --local_dir /var/log/ --remote_dir /store/monitor.ny1.prod/backup/var/log/--host backup --method rsync

It makes log files in /tmp and is capable of emailing as well

#!/usr/bin/perl
 
# You'll see why I did this later on (in case I change the package name I can easily change it back)
use constant PACKAGE => doCronBackup;
 
package PACKAGE;
 
use strict;
use warnings;
 
use Getopt::Long;
 
# Global variable for the command line
# options available to the script
my %g_cmdOptions = (
                        'method'                => "rsync",                                     # Method = rsync or rdiff-backup. Defaults to rsync since that is standard on systems
                        'host'                  => undef,                                       # The host as defined in the ssh_config file
                        'local_dir'             => undef,                                       # The local directory we are backing up
                        'remote_dir'            => undef,                                       # The directory we are putting it to 
                        'success_email'         => "prod_systems\@fixflyer.com",                # The email to send success emails to 
                        'failure_email'         => "sysops\@fixflyer.com",                      # The email to send failure emails to
                        'extra_opts'            => undef,                                       # Extra options we want to pass to rsync or rdiff-backup (this is not checked)
                );                      
 
 
# This will be later populated by init when we obtain the local_dir value and method value
# eg: /etc/httpd with rsync will give us rsync.etc.httpd.log in /tmp
my $g_logFile = undef;
my $g_hostname = "";
 
my $g_useRsync = 0;
my $g_useRdiffBackup = 0;
 
# Depending on the value specified in method command line option, this will be either the abs path to rsync or rdiff-backup
my $g_backupBin = undef;
 
sub getLogTime {
        my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime( time );
        my $date_time = sprintf( "%4d%02d%02d-%02d%02d.%02d",$year+1900,$mon+1,$mday,$hour,$min,$sec );
        return $date_time;
}
 
 
sub print( ) {
 
        print "[ " . &getLogTime() . " ] " . $_[0] . "\n";
 
        if( defined $g_logFile ) {
                if( !open( FILE," >>$g_logFile" ) ) {
                        warn( "! Can not open() log file ${g_logFile}: at line " . __LINE__ . "\n" );
                }
 
                CORE::print( FILE "[ " . &getLogTime() . " ] " .$_[0] . "\n" );
                close( FILE );  
        }
}
 
sub exec( ) {
        my $strCommand = $_[0];
        use IPC::Open3;
        use Symbol qw( gensym );        
        use IO::File;
        local *OUT = IO::File->new_tmpfile;
        local *ERR = IO::File->new_tmpfile;
        my $iPID = open3( gensym, ">&OUT", ">&ERR", "$strCommand" );
        waitpid( $iPID, 0 );
        $? = $? >> 8;
        my $iRetval = $?;
        seek $_, 0, 0 for \*OUT, \*ERR;
        while( <ERR> ) {
                chomp( $_ );
                my $txt = "err: " .  $_;
                &print( "$txt" );
        }
        while( <OUT> ) {
                chomp( $_ );
                my $txt = "out: " . $_;
                &print( "$txt" );
        }
 
        return $iRetval;
}
# doInit(): Used to do the initialization of the script. Get options, do basic checks on globals and command options
sub init( ) {
        GetOptions(
                        "method=s"              =>      \$g_cmdOptions{ 'method' },
                        "host=s"                =>      \$g_cmdOptions{ 'host' },
                        "local_dir=s"           =>      \$g_cmdOptions{ 'local_dir' },
                        "remote_dir=s"          =>      \$g_cmdOptions{ 'remote_dir' },
                        "success_email=s"       =>      \$g_cmdOptions{ 'success_email' },
                        "failure_email=s"       =>      \$g_cmdOptions{ 'failure_email' },
                        "extra_opts=s"          =>      \$g_cmdOptions{ 'extra_opts' }
        );
 
        # Check for empty values
        while( ( my $key, my $value ) = each( %g_cmdOptions ) ) {
                # We dont care about the extra_opts value. The utility (rsync or rdiff-backup) will fail if the user supplies a bad one
                if( $key eq "extra_opts" ) {
                        if( defined $g_cmdOptions{'extra_opts'} ) {
                                next;
                        }
                        else {
                                $g_cmdOptions{'extra_opts'} = "";
                        }
                        next;
                }
 
                if( not defined $value ) {
                        &print( "! Command line option $key is empty. Please supply a value" );
                        exit 5;
                }       
 
        }
 
        # Construct the log file name and delete it if it exists
        my $logFileBaseDir = "/tmp";
        my $logFileName = $g_cmdOptions{ 'method' } . $g_cmdOptions{ 'local_dir'};
        $logFileName =~ tr/\//./;
        $g_logFile = $logFileBaseDir . "/" . $logFileName . ".log";     
        &exec( "rm -v " . $g_logFile ) if -f $g_logFile;
        &print( " + Log file name \t\t=>\t\t $g_logFile" );
 
 
        $g_hostname = `hostname`;
        chomp( $g_hostname );
 
        # Show the values for what we have been supplied
        while( ( my $key, my $value ) = each( %g_cmdOptions ) ) {
                # I hate my OCD 
                if ( $key eq "method" or $key eq "host" ) {
                        &print( " + $key \t\t\t=>\t\t $value " );
                }
                else {
                        $value = "- none specified -" if not defined $value;
                        &print( " + $key \t\t=>\t\t $value " );
                }
        }
 
        # Check that we have our backup bin (rsync or rdiff-backup) available
        if( $g_cmdOptions{ 'method' } eq "rsync" ) {
                $g_backupBin = `sh -c "command -v rsync"`;
                chomp( $g_backupBin );
                if( not defined $g_backupBin or $g_backupBin eq "" ) {
                        &print( "Can not get rsync bin. Please check your PATH and that the command exists" );
                        exit 255;
                }
                &print( " + Backup command \t\t=>\t\t $g_backupBin" );
                $g_useRsync = 1;
        }
        elsif( $g_cmdOptions{ 'method' } eq "rdiff-backup" ) {
                $g_backupBin = `sh -c "command -v rdiff-backup"`;
                chomp( $g_backupBin );
                if( not defined $g_backupBin or $g_backupBin eq "" ) {
                        &print( "Can not get rdiff-backup bin. Please check your PATH and that the command exists" );
                        exit 255;
                }
                &print( " + Backup command \t\t=>\t\t $g_backupBin" );
                $g_useRdiffBackup = 1;
        }
        else {
                &print( "! Invalid backup method supplied.  Valid values are 'rsync' or 'rdiff-backup' (without single quotes). Was supplied " . $g_cmdOptions{ 'method' } );
        }
 
        &print( " + Hostname \t\t=>\t\t $g_hostname" );
        &print( " + Script \t\t\t=>\t\t " . __FILE__ );
}
 
sub execute( ) {
        if( $g_useRsync ) {
 
                &print( "* We are using rsync" );
                my $rsyncDefaultArgs = "-Rakv";
                my $rsyncCommand = "${g_backupBin} ${rsyncDefaultArgs} " . $g_cmdOptions{ 'extra_opts' } . " " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'host' }.":".$g_cmdOptions{ 'remote_dir' };
                &print( "Command to run: " . $rsyncCommand );
                my $rsyncRetval = &exec( $rsyncCommand );
                if( $rsyncRetval != 0 ) {
                        &print( "! Errors running rsync" );
                        my $emailCommand = "cat $g_logFile | mail -s \"ERRORS Encountered: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'failure_email'} . "";
                        &print( "+ Sending email: " . $emailCommand );
                        &exec( $emailCommand );
 
                } else {
                        &print( "+ Succesfully executed rsync" );
                        my $emailCommand = "cat $g_logFile | mail -s \"Success: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'success_email'} . "";
                        &print( "+ Sending email: " . $emailCommand );
                        &exec( $emailCommand );
                }
 
 
        }
        elsif( $g_useRdiffBackup ) {
 
                &print( "* We are using rdiff-backup" );
                my $rdiffDefaultArgs = "--terminal-verbosity=7 --force";
                my $rdiffCommand = "${g_backupBin} ${rdiffDefaultArgs} " . $g_cmdOptions{ 'extra_opts' } . "  " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'host'} . "::" . $g_cmdOptions{ 'remote_dir' };
                &print( "Command to run: " . $rdiffCommand );
                my $rdiffRetval = &exec( $rdiffCommand );
                if( $rdiffRetval != 0 ) {
                        &print( "! Error running rdiff-backup. Please view the logs - return code was: $rdiffRetval" );
                        # Send email to error list
                        my $emailCommand = "cat $g_logFile | mail -s \"ERRORS Encountered: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'failure_email'} . "";
                        &print( "+ Sending email: " . $emailCommand );
                        &exec( $emailCommand );
 
                } else {
                        &print( "+ Successfully executed rdiff-backup" );
                        my $emailCommand = "cat $g_logFile | mail -s \"Success: $g_hostname " . $g_cmdOptions{ 'local_dir' } . " " . $g_cmdOptions{ 'method' } . " Job\" " . $g_cmdOptions{ 'success_email'} . "";
                        &print( "+ Sending email: " . $emailCommand );
                        &exec( $emailCommand );
                }
 
 
                # And exit with the return value of rdiff-backup
                exit $rdiffRetval;
 
        } else {
                &print( "You shouldn't be here, silly" );
                exit 155;
        }
 
}       
 
&init( );
&execute( );

Sphere: Related Content

Category Categories: Computers, Linux/Unix, Networks/Servers, Ubuntu | Tag Tags: backup script, backup with cron using perl, cron backup script, generic cron, generic perl script, perl backup, perl backup script | No Comments

[Bash/Linux] POC – Rip YouTube Playlist

October 7, 2011 | Posted by Chuck Kozler

This is just a POC now and is just the sloppy scripts. I will soon rewrite this to be more elegant, safe, and clean and as well provide RPM, DEB, and AUR packages

Dependencies: youtube-dl, lame, faad

Debian/Ubuntu:

sudo apt-get install youtube-dl lame faad

Arch:

sudo pacman -Sy youtube-dl lame faad

Not sure about CentOS/RedHat/Fedora but Im sure its youtube-dl through yum

In either case, here is how to use it

$ ./youtubeBatch.sh http://www.youtube.com/playlist?list=PLC28FD8A0C619191E

Which will initialize the download. Afterwards I would suggest moving all of the files

$ mkdir files/
$ find . -maxdepth 1 -type f -iname \*.aac | xargs -I{} mv {} files/

Then run the convert script

$ ./convert.sh files/

This will convert all of the aac files to wav and then wav to mp3

youtubeBatch.sh

 
#!/bin/bash
 
if [ -z "$1" ]; then echo "!! You must enter a playlist URL to download"; exit 5; fi
 
PLAYLIST_URL="$1"
 
 
wget -O /tmp/playlist.output ${PLAYLIST_URL} 
 
for i in `cat /tmp/playlist.output | grep --color=auto -E 'watch\?v=' | cut -d '=' -f3 | sed 's/&amp;list//g' | grep -iv "hid"`; do
	URL="http://www.youtube.com/watch?v=$i"
	youtube-dl --extract-audio  ${URL}
done

convert.sh

#!/bin/bash
 
if [ -z "$1" ]; then echo "!! ERROR: You must supply a directory with a list of files to convert"; exit 5; fi
 
IN_DIR="$1"
 
mkdir -p ${IN_DIR}/mp3s/
 
for i in `ls ${IN_DIR}`; do
 
	cd ${IN_DIR}
	FILENAME_NO_EXT=`echo ${i} | cut -d '.' -f1`
 
	echo "+ Running faad on $i (not ext = ${FILENAME_NO_EXT}"
 
	faad $i 
	if [ $? -ne 0 ]; then echo "!! ERROR running faad on $i"; continue; fi
 
	echo "+++ RUNNING LAME"
	lame ${FILENAME_NO_EXT}.wav mp3s/${FILENAME_NO_EXT}.mp3
 
done

Sphere: Related Content

Category Categories: Computers, Linux/Unix, Programming, Ubuntu | Tag Tags: youtube, youtube convert playlist, youtube mp3 linux, youtube mp3 playlist, youtube mp3 playlist rip linux, youtube rip, youtube rip linux, youtube rip playilst and make mp3, youtube rip playlist, youtube save playlist, youtube ubuntu mp3 | 2 Comments

[MySQL] Backup all MySQL databases

August 3, 2011 | Posted by Chuck Kozler

Just a quick and dirty post. I needed to do this at work and figured someone else would find it useful.

As a suggestion, you can create a read-only user that executes the backup and temporarily not have it supply a password

for i in `mysql -u root -p -e 'show databases' -s --skip-column-names`; do mysqldump -u root -p $i > $i.sql; done

Sphere: Related Content

Category Categories: Computers, Linux/Unix, Networks/Servers | Tag Tags: backup, backup all mysql databases, backup mysql, how to backup all mysql databases one command, MySQL, mysql backup | No Comments

[FreeBSD] Nightly ZFS Snapshot Script

June 29, 2011 | Posted by Chuck Kozler

Hey Guys,

Nothing special here. This is just a basic script that will loop your ZFS shares skipping anything in the exclude list. The exclude list works by creating a regular expression for skipping share names listed in zfs list– my users home directories (their private backups) are snapshotted by a different script. Anywho, pretty self explanatory with the basic procedure of:

1.) Get a list of the ZFS volumes we are concerned with by first creating our exclusion list then passing it to grep -iv and then grepping the pool we’re concerned with. This populates our list of ZFS volumes to work with
2.) Get the current day of week
3.) Loop our ZFS volumes obtained from getZVols and attempt to create the snapshot based on ZFS volume name @ day of week (eg: zpool/hotbackup@monday)
4.) If we hit one that already exists we destroy it then recreate it again

The destroy works as an automatic roll-over whereas I run this nightly and only care about the previous week because each week I bzip2 the data and move it somewhere else.

#!/usr/local/bin/bash
#########################################################################################
LOG_FILE="/tmp/`basename $0`.log"                                                       #
echo -e "\n\n" && cat /dev/null > ${LOG_FILE}; echo -e "\n----\n" >> ${LOG_FILE}        #
GOT_ERROR=0                                                                             #
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/root/bin  #
export PATH                                                                             #
#########################################################################################
# Script to snapshot everything else except user home directories and 
# primary zpool root directory
 
 
 
# Pool name as defined when you run zpool create [...]
POOL_NAME="zpool"
 
# Exclusion list of directories that we dont want to snapshot
# Our users home directories are taken care of by a separate script
declare -a EXCLUDE_DIR_LIST=( 
				"home"
			)
 
 
 
 
function write { 
	echo -e "$1" >> ${LOG_FILE}
	echo -e "$1"
}
 
# Holy craptastical bash hackery
function makeExcludePattern {
 
	declare -a EXCLUDE_LIST=($@)
 
	# If we get zero we have to return something since we're passing this to grep.  I generate a random bash number
	# and then pass it to MD5 for even more randomness, clip it to 8 characters, and add the grep regex
	if [ $# -eq 0 ];
	then
		echo "!! Got a zero value on exclude list.  Returning a random string to pass to exclude" >> ${LOG_FILE}
		echo "${RANDOM}" | md5 | cut -c1-8 | xargs -I{} echo "\({}\)"
		return
	fi
 
	# If we get one argument then we just parse it and leave the function
	if [ $# -eq 1 ]; 
	then
		echo "!! Got a single value in exclude list" >> ${LOG_FILE}
		echo "\(${EXCLUDE_LIST[@]}\)"
		return
	fi
 
	# Otherwise we parse the entire exclude list and echo it back
	echo "makeExcludeList: ${EXCLUDE_LIST[@]}" >> ${LOG_FILE}
	echo "\(${EXCLUDE_LIST[@]}\)" | sed 's/ /\\|/g' 
}
 
# Get the exclusion list and then pass it to grep
function getZVols {
	EXCLUDE_DIRS=`makeExcludePattern ${EXCLUDE_DIR_LIST[@]}`
	zfs list | grep -iv "${EXCLUDE_DIRS}" | grep ${POOL_NAME} | awk '{ print $1 }'
}
 
function getDayOfWeek {
	date +%A | tr '[A-Z]' '[a-z]'
}
 
# Get day of week as soon as we run 
DAY_OF_WEEK=`getDayOfWeek`
 
for i in `getZVols`; 
do
	ZFS_VOLUME="$i"
	ZFS_VOLUME_SNAPSHOT="${ZFS_VOLUME}@${DAY_OF_WEEK}"
	write " + Working with ZFS volume => ${ZFS_VOLUME}"
	write " \t++ Creating zfs snapshot ${ZFS_VOLUME_SNAPSHOT}"
	zfs snapshot ${ZFS_VOLUME_SNAPSHOT} >> ${LOG_FILE} 2>&1
	if [ $? -eq 1 ]; 
	then
		write " \t\t!! ZFS Volume snapshot appears to exist already.  Destroy existing and recreating"
		zfs destroy ${ZFS_VOLUME_SNAPSHOT} >> ${LOG_FILE} 2>&1
		RETVAL=$?
		if [ ${RETVAL} -eq 0 ]; 
		then
			write " \t\t++ Successfully destroyed snapshot.  Attempting recreation"
			zfs snapshot ${ZFS_VOLUME_SNAPSHOT} 
			RETVAL=$?
			if [ ${RETVAL} -eq 0 ]; 
			then
				write "\t\t\t ++ Successfully created snapshot ${ZFS_VOLUME_SNAPSHOT}"
			else
				write "\t\t\t !! ERROR: Some other return code other than zero on creating snapshot after destroy.  Return code was ${RETVAL}"
				unset ${RETVAL}
				GOT_ERROR=1
			fi
		else
			write " \t\t!! ERROR: Some other return code on attempting to destroy existing snapshot for ${ZFS_VOLUME_SNAPSHOT}.  Return code was ${RETVAL}"
			unset ${RETVAL}
			GOT_ERROR=1
		fi
	elif [ $? -eq 0 ] || [ $? -eq 1 ]; 
	then
		write " \t\t +++ Successfully created ZFS snapshot for ${ZFS_VOLUME_SNAPSHOT}"
	else
		write " \t\t +++ Some other return code.  Assuming error on ${ZFS_VOLUME_SNAPSHOT} for safe keeping"
		GOT_ERROR=1
	fi
done
 
 
if [ ${GOT_ERROR} -eq 1 ];
then
	write "Sending error email"
	cat ${LOG_FILE} | mail -s "ERRORS Encountered: `hostname -s` ${POOL_NAME} Snapshots Job" sysops@fixflyer.com
else
	write "Sending success email"
	cat ${LOG_FILE} | mail -s "Success: `hostname -s` ${POOL_NAME} Snapshots Job" prod_systems@fixflyer.com
fi

Sphere: Related Content

Category Categories: Computers, Home, Linux/Unix | Tag Tags: ZFS, zfs auto backup, zfs automated backup, zfs backup tool, ZFS FreeBSD, zfs nightly backup, zfs script, ZFS Snapshot, ZFS Snapshot script, zfs snapshot utility | No Comments

[FreeBSD] FreeBSD 8.2 Auto Installation with Packages

June 13, 2011 | Posted by Chuck Kozler

Hi Guys,

There is nothing big here, this just my auto-installation script added with DHCP functionality. Changes include:

- Added a modified dhclient binary to allow the use of DHCP without having to drop privileges or add any additional users
- Automatic package installations for vim, sudo, bash, and lynx
- Colored bash consoles (when you login you will have to source /etc/profile)
- Automatic SSH enable at first boot

You can find the script update on my new Git: Click to view

You can find the freebsd.hd file which contains the latest binaries & changes on Rapidshare at: Click to download

Sphere: Related Content

Category Categories: Computers, Home, Linux/Unix, Networks/Servers, Projects | Tag Tags: auto install Freebsd, automated freebsd, freebsd, FreeBSD Auto Install, FreeBSD init Install, FreeBSD Scripted Install, FreeBSD Scripting, how to auto install freebsd, how to automate freebsd install, init install automated freebsd | 3 Comments

[FreeBSD] FreeBSD 8.2 init-based Auto Installation

June 10, 2011 | Posted by Chuck Kozler

EDIT 1 (20110610): I already found one bug with this but perhaps technically two. The first is when I parse the output of sysctl -a kern.disks I do not sort the format so if you have multiple disks (eg: not a single drive or a RAID 1 or 10) then it will install on the last disk rather than the first. The second issue is that I just out right fail if the networking was not successful. That is, I should edit it to get all devices listed in ifconfig output and keep trying until it is successful unless, of course, it reaches the last one and can not continue.

EDIT 2 (20110613): I have added a new post which contains automatic networking through DHCP and a scripted auto-detect as well as sorted ordering of drives to better obtain the primary disk, and finally, automatic package installation

Link

So, fresh off my kick with enjoying a fully operational new PXE Boot environment at work and learning many new things about it, I decided it was time to configure FreeBSD to PXE Boot. This document will not cover how to PXE Boot the regular installer (though I will try to make a note of what you would need to do to use the regular ISO).

That being said, lets get right into it.

I will cover the following:

Prerequisites & Assumptions
A downloadable copy of my PXE-Bootable FreeBSD Auto-Install
Configuration details of my PXE-Bootable FreeBSD Auto-Install
Configuring PXE Linux to boot PXE-Bootable FreeBSD Auto-Install
Instructions on how to modify my install to suit your environment

Also, any references that you do not understand will come later in my detailed explanation of how to modify the auto-install script to set your parameters. For instance, I mention paths inside of mfsroot.gz but you may not understand why when you gunzip mfsroot why you see no paths– I will cover it all later.

If somethings are not clear here please go to my post about setting up PXELinux to PXE Boot Windows 7 — that document also explains some items regarding PXE setup and may help you in this document

http://ckozler.net/?p=228

Prerequisites & Assumptions

The prerequisites and assumptions to this are as follow

An already configured PXE Environment with memdisk that can boot ISO’s and harddisk images and supports options “iso” & “harddisk”
We are using the whole disk that you are installing to when you run the auto-installer
Installing the GENERIC FreeBSD Kernel
The compressed files (in this case, .tar.gz) base & kernel files are contained in a directory named “base” and “kernels” respectively inside the compressed files. That is, the base.tar.gz file we download has a single directory of “base/” which was a tar -cvzf base.tar.gz base/ from the FreeBSD installation media. Likewise with the kernels.tar.gz file
The above mentioned files located on an accessible FTP and/or an FTP configured and ready to be used as such
That the FTP Host can be accessed via IP as hostname support is not yet built into this (though you can change it if you really need to and/or know how to)

My Preconfigured PXE-Bootable FreeBSD AutoInstaller

You can download mine from here (this is RapidShare but only 60MB)

Download freebsd.hd

What is this? – Configuration Details

What this is pretty simple– it is a modified boot-only.iso from the FreeBSD website which inside contains a modified mfsroot.gz. This mfsroot.gz is the filesystem that is loaded by the boot-only ISO and is modified to include needed tools like cut, ping, mount, and the similar. Without kicking off the init script, this would be a pretty good network-bootable FreeBSD rescue system.

When using the boot-only ISO the boot process goes like this:

Boot ->load ISO -> load boot record from /boot/mbr -> load kernel from /boot/kernel -> load /boot/loader.conf and read settings -> loader.conf says to set the root to mfsroot -> mount mfsroot -> init process

We tell our boot-only ISO’s /boot/loader.conf to run our init script which is contained /etc on the mfsroot.gz file system.

freebsd.hd — /boot/loader.conf

verbose_loading="YES"
mfsroot_load="YES"
mfsroot_type="mfs_root"
mfsroot_name="/boot/mfsroot"
 
#
# Added to load a custom init scripted install
#
init_path="/sbin/init"
init_script="/etc/rc.jumpstart"
init_shell="/bin/sh"

You can see where I added the modifications to boot our /etc/rc.jumpstart file and commenting out the sysinstall init.

At a high level, our /etc/rc.jumpstart does the following (the script can be seen later)

Auto detecting network the best it can– this facility is rudimentary at best by parsing ifconfig and getting the first device it sees. This can probably be expanded to try each device until it gets an active link
Pings the host as defined in the FTP_HOST variable
Auto detecting the disk to write to the best it can– again, this facility is rudimentary. It executes sysctl -a and gets the kern.disks output and grabs the first disk it sees then uses that for slicing and labeling
Automatically formats and slices the disk (using fdisk)
Automatically creates the label and makes the drives bootable
Downloads the core & kernel files from an FTP– this will probably be where you have to change the script
Mounts the devices and installs the base & kernel packages
Cleans up
Reboots

This is the script which can be located in the mfsroot.gz /etc/ directory. I believe I have commented it well enough and identified the fields you most likely need to change

#!/bin/sh
 
#
# The IP to assign once we boot and find out primary network device
#
MY_IP="192.168.1.250"
 
#
# FTP information
#
# This should be about the only information you will really ever
# need to change as most of the other stuff relies on specific
# static things that you shouldnt change unless you know what you're
# doing
 
# FTP Host - IP/username
FTP_HOST="192.168.1.21"
FTP_DIR="/FreeBSD/8.2-RELEASE/"
FTP_BASE_FILES="base.tar.gz"
FTP_KERNEL_FILES="kernels.tar.gz"
 
FTP_BASE_LINK="ftp://${FTP_HOST}${FTP_DIR}${FTP_BASE_FILES}"
FTP_KERNEL_LINK="ftp://${FTP_HOST}${FTP_DIR}${FTP_KERNEL_FILES}"
 
# The mount point on the booted to mfsroot for placing base &amp; kernel files
# The variables are the same but the install.sh for base &amp; kernel files
# need the variable DESTDIR available to them so they know where to set the files
ROOT_TMP_MOUNT="/mnt"
DESTDIR="${ROOT_TMP_MOUNT}"
 
# This should always default to /dev/md0
MFSROOT_DEV="/dev/md0"
 
# Obtain the primary network device.
# This is rudimentary at best and will obtain the first device
# Listed in ifconfig
NETDEV=`ifconfig | head -n 1 | cut -d ':' -f1`
 
# Obtain the primary disk
# Again, rudimentary at best.
DISK=`sysctl -a | grep -i kern.disks | head -n 1 | awk '{ print $2 }'`
DISK="/dev/${DISK}"
 
# Set the WORKING_DISK variable to be ${DISK}s1 for slice 1
# We assume s1 because we use fdisk -I to use the whole disk and make a single
# slice
WORKING_DISK="${DISK}s1"
 
# Paritions below are set through bsdlabel file that we use.
 
# Set the root partition
PART_ROOT="${WORKING_DISK}a"
 
# Set the swap partition
PART_SWAP="${WORKING_DISK}b"
 
# Set the usr partition
PART_USR="${WORKING_DISK}d"
 
# Set the "extra" storage parititon
PART_STORE="${WORKING_DISK}e"
 
##########################
# Do not edit below these lines unless you need to
##
sysctl kern.geom.debugflags=16
echo "*************************************"
echo "*     init based auto-install       *"
echo "*************************************"
echo
echo
echo "Starting init based auto-install"
echo
echo
 
echo " ! Making ${MFSROOT_DEV} read-write for temporary file storage"
mount -o rw /dev/md0 /
if [ $? -ne 0 ]; then echo " ** There was an error mounting ${MFSROOT_DEV}"; exit 1; fi
 
# Set IP address and let the device settle before continuing on
echo " ! Setting IP address"
ifconfig ${NETDEV} ${MY_IP}
sleep 10
# Try to ping the FTP host before we go anywhere
ping -c 3 ${FTP_HOST}
if [ $? -ne 0 ]; then echo " ** Could not ping FTP host. FTP Host is either down or network configuration failed"; exit 1; fi
 
echo " ! Clearing existing partition table from ${DISK}"
dd if=/dev/zero of=${DISK} count=50
sleep 3
 
echo " ! Executing fdisk to create slice and install boot loader"
fdisk -BI -a -b /boot/mbr ${DISK}
sleep 3
 
echo " ! Writing labels"
######################################
# ONLY CHANGE THIS IF YOU ABSOLUTELY #
# NEED TO. IF YOU DO, YOU MAY HAVE   #
# TO CHANGE THE VARIABLES ABOVE      #
# FOR THE PARTITIONS                 #
######################################
BSDLABEL_FILE="/var/tmp/bsdLabelFile"
 
# a = root |  b = swap | c = unused
# d = usr  |  e = everything else
echo "
a:     10G      16      4.2BSD  0       0       0
b:      4G       *      swap    0       0       0
c:       *       *      unused  0       0
d:     10G       *      4.2BSD  0       0
e:       *       *      4.2BSD  0       0
" &gt; ${BSDLABEL_FILE}
bsdlabel -R ${WORKING_DISK} ${BSDLABEL_FILE}
bsdlabel -B -b /boot/boot ${WORKING_DISK}
sleep 3
 
echo " ! Running newfs on partitions"
echo " !! Making root partition on ${PART_ROOT}"
newfs ${PART_ROOT}
echo " !! Making usr partition on ${PART_USR}"
newfs ${PART_USR}
echo " !! Making store partition on ${PART_STORE}"
 
echo " ! Mounting temporary root for beginning installation of files"
mount ${PART_ROOT} ${ROOT_TMP_MOUNT}
echo " ! Making usr partition on temporary root"
mkdir -p ${ROOT_TMP_MOUNT}/usr
echo " ! Mounting ${PART_USR} on ${ROOT_TMP_MOUNT}/usr for installation of files"
mount ${PART_USR} ${ROOT_TMP_MOUNT}/usr
 
echo " ! Changing into temporary root to begin install"
cd ${ROOT_TMP_MOUNT}
 
echo " ! Fetching base files from ${FTP_BASE_LINK}"
ftp ${FTP_BASE_LINK}
 
echo " ! Unpacking files from ${FTP_BASE_FILES}"
tar -xvzf ${ROOT_TMP_MOUNT}/${FTP_BASE_FILES}
echo " ! Changing into base files directory"
cd ${ROOT_TMP_MOUNT}/base
echo " ! Beginning install of base files"
export DESTDIR
echo "y" | sh install.sh
 
echo " ! Leaving base directory"
cd ${ROOT_TMP_MOUNT}
 
echo " ! Fetching kernel files to begin install"
ftp ${FTP_KERNEL_LINK}
 
echo " ! Unpacking files from ${FTP_KERNEL_FILES}"
tar -xvzf ${ROOT_TMP_MOUNT}/${FTP_KERNEL_FILES}
 
echo " ! Changing into kernel files directory"
cd ${ROOT_TMP_MOUNT}/kernels
echo " ! Beginning install of kernel files"
export DESTDIR
echo "y" | sh install.sh GENERIC
 
echo " ! Writing /etc/fstab"
 
echo -e "${PART_ROOT} \t\t / \t\t ufs \t rw \t\t 1 \t\t 1" &gt; ${ROOT_TMP_MOUNT}/etc/fstab
echo -e "${PART_SWAP} \t\t none \t\t swap \t sw \t\t 0 \t\t 0" &gt;&gt; ${ROOT_TMP_MOUNT}/etc/fstab
echo -e "${PART_USR} \t\t /usr \t\t ufs \t rw \t\t 1 \t\t 1" &gt;&gt; ${ROOT_TMP_MOUNT}/etc/fstab
 
# This could probably just be a copy over from GENERIC but a symlink works just as well
echo " ! Writing symlinks /boot/kernel -&gt; /boot/GENERIC"
echo " !! Changing into ${ROOT_TMP_MOUNT}/boot"
cd ${ROOT_TMP_MOUNT}/boot
echo " !! Deleting kernel directory"
rm -rfv kernel
echo " !! Setting symlink"
ln -s GENERIC kernel
 
# Write out build date &amp; time so we know we have the right stuff
BUILD_DATE=`date +%Y%m%d`
BUILD_TIME=`date +%H%M.%S`
echo "Build date % time ${BUILD_DATE} @ ${BUILD_TIME}" &gt; ${ROOT_TMP_MOUNT}/build.info
 
echo " ! Beginning cleanup"
echo " !! Changing into temporary root directory"
cd ${ROOT_TMP_MOUNT}
echo " !! Deleting downloaded files (base)"
rm -rfv base ${FTP_BASE_FILES}
echo " !! Deleting downloaded files (kernels)"
rm -rfv kernels ${FTP_KERNEL_FILES}
echo " !! Changing into /"
cd /
echo " !! Unmounting temporary usr file system"
umount ${ROOT_TMP_MOUNT}/usr
echo " !! Unmounting temporary root file system"
umount ${ROOT_TMP_MOUNT}
 
echo
echo
echo " *** REBOOTING ***"
echo
echo
 
sleep 5
reboot

And here is a video of it in action

Configuring PXE Linux to Boot FreeBSD & FreeBSD Auto Install

We are already under the assumption you have a working PXE Linux install and have successfully PXE booted other OSs. You will now need to copy the above mentioned freebsd.hd and place it in your TFTP directory. Below you can see an image of my configuration relative to that of booting the FreeBSD hard disk image with memdisk.

Regular FreeBSD Manual Install

In the above picture you can see my menu entry for “FreeBSD 8.2 (64-bit Manual)” which is just the boot-only ISO from the FreeBSD website downloaded and placed into the respective path that you see in the APPEND section.

My above configuration says to load memdisk from images/FreeBSD/x86_64/8.2/memdisk though this may be different for you so change accordingly. It then says to APPEND the initrd of images/FreeBSD/x86_64/8.2/freebsd_amd64.iso which, as previously stated, is nothing more than the FreeBSD boot-only ISO renamed and placed in that path. The changes you need to make are to edit the path to memdisk and the path to your boot-only ISO

My FreeBSD AutoInstall

The second section of that picture is the auto install. As stated in the above section, you will need to change your the paths of the file relative to your TFTP directory for where you have your memdisk and my downloaded freebsd.hd file

Modifying the AutoInstall for Your Environment

This is the tricky part and takes a number of steps plus an extra FreeBSD machine.

Please note that I use sudo but all of these commands below assume you already root.

First we need to setup our building enviornment. I would suggest doing this to keep everything organized.

$ mkdir -p ~/autoinstall/{build,files}
$ mkdir -p ~/autoinstall/build/{hd.mnt,mfsroot.mnt}

This will make the following directory structure

~/autoinstall/
~/autoinstall/build
~/autoinstall/build/hd.mnt
~/autoinstall/build/mfsroot.mnt
~/autoinstall/files

You will want to copy the downloaded freebsd.hd file into ~/autoinstall/files. Now, we are going to mount the hard disk image so we can extract my already modified mfsroot.gz. This uses a combination of mdconfig & mount. mdconfig is used to give it a block device so that you can mount it and mount it read/write. First I will show you what to do and then I will show you what my output/example looked like

Note: Set ${#} to whatever is outputted from the mdconfig command. — see my example below

$ mdconfig -a -t vnode -f ~/autoinstall/files/freebsd.hd
$ mount /dev/md${#}a ~/autoinstall/build/hd.mnt
$ ls ~/autoinstall/build/hd.mnt/

And my example:

As you can see, you should have a boot directory and a cdrom.inf file.

Now lets copy the mfsroot.gz file from ~/autoinstall/build/boot/ into our files directory for storage

$ cp -v ~/autoinstall/build/hd.mnt/boot/mfsroot.gz ~/autoinstall/files/

Now that we have our mfsroot.gz file, we need to gunzip it, create an md device with mdconfig, and then subsequently mount it. Again, I will display what you should type and then show you my example

Note: Set ${#} to whatever is outputted from the mdconfig command. — see my example below

$ gunzip -v ~/autoinstall/files/mfsroot.gz
$ mdconfig -a -t vnode -f ~/autoinstall/files/mfsroot
$ mount /dev/md${#} ~/autoinstall/build/mfsroot.mnt
$ ls ~/autoinstall/build/mfsroot.mnt/

Now we have everything open that we need to open in read/write and we can make our changes as required.

As I previously stated, the init script is in /etc/ under the name “rc.jumpstart”. The sections you should most likely change, at a minimum, are as follow:

The IP the default network device gets. Change what is in the ” ” to an available IP on your network. This should be an IP that is on the same subnet of that of your FTP (preferably but not required)

#
# The IP to assign once we boot and find out primary network device
#
MY_IP="192.168.1.250"

The FTP information where you will obtain the compressed base and kernel files

# FTP Host - IP/username
FTP_HOST="192.168.1.21"
FTP_DIR="/FreeBSD/8.2-RELEASE/"
FTP_BASE_FILES="base.tar.gz"
FTP_KERNEL_FILES="kernels.tar.gz"

h2. Advanced Configuration

This isnt technically “advanced” configuration but changing this allows you to make the script not guess what it should use for network and disk devices. If you know exactly what FreeBSD will create in /dev once you boot to the network installer then you should change this to avoid any issues but otherwise, leave it up to the script to try to do it.

Network Configuration — as an example I use “em0″ as the device because I know FreeBSD will create that for VirtualBox (where I tested this initially). If I wanted to take the guess work out I would do the following.

Edit:

# Obtain the primary network device.
# This is rudimentary at best and will obtain the first device
# Listed in ifconfig
NETDEV=`ifconfig | head -n 1 | cut -d ':' -f1`

And make it look as such:

# Obtain the primary network device.
# This is rudimentary at best and will obtain the first device
# Listed in ifconfig
#NETDEV=`ifconfig | head -n 1 | cut -d ':' -f1`
NETDEV="em0"

Disk Configuration — as an example I use ad0 as the device because I know FreeBSD will create that for VirtualBox (where I tested this initially).

Edit

# Obtain the primary disk
# Again, rudimentary at best.
DISK=`sysctl -a | grep -i kern.disks | head -n 1 | awk '{ print $2 }'`
DISK="/dev/${DISK}"

And make it look as such:

# Obtain the primary disk
# Again, rudimentary at best.
#DISK=`sysctl -a | grep -i kern.disks | head -n 1 | awk '{ print $2 }'`
#DISK="/dev/${DISK}"
DISK="/dev/ad0"

While the absolute path is not really required since fdisk and bsdlabel will attempt to auto-resolve the path for the supplied disk name, I would highly advise in passing an absolute path there.

All other parameters of the script are easily changed but you should know what you’re doing and how it will impact the install.

Now that we have made all of our necessary changes, we want to package everything up.

First, unmount the mfsroot device and delete the md device so we can commit our changes

Note: Remember when I ran mdconfig I was assigned 1 — yours may be different

$ umount ~/autoinstall/build/mfsroot.mnt
$ mdconfig -d -u 1

Now that we have the changes committed and successfully unmounted we can gzip back our mfsroot file. Once it is gzipped we are going to copy it back to our bootable FreeBSD hard disk image

$ gzip -vk ~/autoinstall/files/mfsroot
$ cp -fv ~/autoinstall/files/mfsroot.gz ~/autoinstall/build/hd.mnt/boot/mfsroot.gz

Now lets unmount the hard disk image

Note: mdconfig assigned me 0 so when you execute mdconfig, make sure to pass the value you were assigned above when creating the md device

$ umount ~/autoinstall/build/hd.mnt
$ mdconfig -d -u 0

Now we have successfully modified the boot image. Now copy this back to your PXE server overwriting the old one (or not and changing your path respectively in your PXE boot file) and when you boot your auto installer utilize the newest changes to your rc.jumpstart file.

Please ask any questions if you have any!

Sphere: Related Content

Category Categories: Computers, Linux/Unix, Projects | Tag Tags: auto install Freebsd, automate freebsd installation, automated freebsd, automatic installation unix, freebsd, FreeBSD Auto Install, FreeBSD init Install, FreeBSD Scripted Install, FreeBSD Scripting, how to auto install freebsd, how to automate freebsd install, init install automated freebsd, sysinstall automated, unix | 2 Comments

[Linux] Windows 7 PXE Boot From Linux

June 3, 2011 | Posted by Chuck Kozler

EDIT 1 (20110604): Fixed the WAIK Link
EDIT 2 (20110606): It appears that even though I generated and have successfully booted from the AMD64 WinPE image, on newer hardware I reach the error of “Subsystem needed to support the image type is not present”. This was actually resolved by just booting from the X86.iso. That being said, I am making changes below to remove generation for the amd64.iso and making the necessary changes to pictures and such. Old data will be struck through so it is still retained by ignored.

Hi all,

Its been awhile but I finally have a very good post to put out here that could be of some use people. Most of the existing documents and blog posts I have found were helpful in getting me in the right direction but generally lacked good detail on how exactly to achieve this.

This document is going to be continuously updated to ensure the details are as clear as possible and all of the little nuances are covered.

The items we are going to cover are:

1. Installing TFTP & PXELinux on CentOS 5.6
2. Setting up PXELinux for booting WinPE
3. Setting up Samba and the files to use to install Windows 7
4. Creating a WinPE disk image to boot (both AMD64 & x86)
5. Achieving the boot process
6. How to mount Samba share when in WinPE and Install Windows 7
7. An extra note on how to do this with Windows 7 in VirtualBox

The assumptions we make through this are:

1. We have DHCP enabled on our network
2. We are not in an Active Directory or Domain Structure — I only make this assumption because where I did this we currently do not have a Domain System setup so the authentication could be substantially different
3. You have root access on the PXE server
4. That you have an extra Windows 7 around that you can do the required Windows tasks on (like generating the WinPE installation disks)

Installing TFTP & PXELinux on CentOS 5.6

The first and most core step. First, install TFTP if you do not have it installed. My PXE server was installed from a slim installation of CentOS through KickStart so it did not have the TFTP server installed

[pxe_server]$ yum install tftp

Once TFTP is installed we need to edit xinetd.d’s definition (/etc/xinetd.d/tftp) for TFTP to enable it

You will see it displayed as this

Modify the parameter of “disable” and set it to “no” as seen below

The installation of TFTP will make the default directory in /tftpboot

Restart xinetd

[pxe_server]$ service xinetd restart

TFTP will now be listening via xinetd for connections

Setting up PXELinux for booting WinPE

Now, as previously stated, I am running a slimed down version of CentOS so I also needed to install syslinux package too.

Note that this installs 3.11 and we will need to update the memdisk boot file to facilitate booting from ISO later on (it will be covered)

[pxe_server]$ yum install syslinux

Some of the directions below were copied and pasted from the CentOS how-to on PXE Booting setup to save me some time and to make it match my setup.

This will copy over all of the files from the syslinux package and place them into our TFTP directory that TFTP knows to serve up through its xinetd.d definition

[pxe_server]$ cp /usr/lib/syslinux/pxelinux.0 /tftpboot
[pxe_server]$ cp /usr/lib/syslinux/menu.c32 /tftpboot
[pxe_server]$ cp /usr/lib/syslinux/memdisk /tftpboot
[pxe_server]$ cp /usr/lib/syslinux/mboot.c32 /tftpboot
[pxe_server]$ cp /usr/lib/syslinux/chain.c32 /tftpboot

pxelinux.0: How we will boot
menu.c32: To display a multilevel color text based menu to the user (we will use this one)
memdisk: We will be using this to boot Windows. Mostly used to boot anything that is not Linux and/or legacy systems — memdisk from SysLinux Webpage
mboot.c32: Best explanation is here — mboot.c32 from SysLinux Webpage
chain.c32: Best explanation is here — chain.c32 from SysLinux Webapge

Now we need to create the pxelinux.cfg directory. You must use this directory name from the TFTP root directory because that is how the pxelinux.0 file knows where to look for the menu information.

[pxe_server]$ mkdir /tftpboot/pxelinux.cfg

Now lets create our boot directory structure. The below is an example and can be adjusted as you see fit. I like mine organized by /tftpboot/images/OS/ARCH/ as seen below:

I know this can be consolidated with a { .. } and -p but I am doing it line by line to show the hierarchy

Remember, these are not the distribution files. These are architecture dependent boot files that you will need entries for in the menu ( to be covered later )

[pxe_server]$ mkdir /tftpboot/images/
[pxe_server]$ mkdir /tftpboot/images/Windows_7
[pxe_server]$ mkdir /tftpboot/images/Windows_7/x86
[pxe_server]$ mkdir /tftpboot/images/Windows_7/x86_64

Upgrade SysLinux memdisk Image

Now, we need to upgrade our SysLinux package so our memdisk boot image can support loading ISO’s

Some other notes here:

1. I had to install the development tools through the groupinstall argument to yum.
2. I had to install nasm package for building the memdisk image
3. I had to install unzip

[pxe_server]$ yum groupinstall 'Development Tools'
[pxe_server]$ yum install unzip
[pxe_server]$ yum install nasm
[pxe_server]$ mkdir /root/syslinux-4.04
[pxe_server]$ cd /root/syslinux-4.04
[pxe_server]$ wget http://www.kernel.org/pub/linux/utils/boot/syslinux/4.xx/syslinux-4.04.zip
[pxe_server]$ unzip syslinux-4.04.zip
[pxe_server]$ cd memdisk
[pxe_server]$ make
[pxe_server]$ cp memdisk /tftpboot/images/Windows_7/x86_64/
[pxe_server]$ cp memdisk /tftpboot/images/Windows_7/x86/

Configure DHCP

Again, with the minimal installation, I had to install DHCP server daemon

[pxe_server]$ yum install dhcp

I will display my file here with the necessary notes that you will (or should) make to facilitate your network.

Most stuff here is semantic but the field you must change here is “next-server” which should be your servers IP. You can toy with everything else later.

[pxe_server]$ vi /etc/dhcpd.conf

ddns-update-style               none;
 
site-option-space               "pxelinux";
option space                    pxelinux;
option space                    pxelinux;
option pxelinux.magic           code 208 = string;
option pxelinux.configfile      code 209 = text;
option pxelinux.pathprefix      code 210 = text;
option pxelinux.reboottime      code 211 = unsigned integer 32;
 
shared-network                  "fixflyer.int"
{
        subnet 192.168.1.0      netmask         255.255.255.0
        {
 
                # Setup your domain name, your routers gateway IP, and
                # any DNS servers you should use by default when DHCP assigns the network to the connecting client.
                # If your using an 192.168.1.0/24 IP subnet then you can
                # probably just remove the first DNS server and leave the rest (maybe change domain name?)
                option          domain-name             "fixflyer.int";
                option          routers                 192.168.1.1;
                option          domain-name-servers     192.168.1.70, 4.2.2.2, 4.2.2.1;
 
                allow                                   bootp;
                allow                                   booting;
                use-host-decl-names                     on;
 
                range                                   192.168.1.230 192.168.1.250;
 
                # This IP for "next-server" should be your servers IP
                next-server                             192.168.1.102;
                filename                                "/pxelinux.0";
 
        }
}

Restart DHCP

[pxe_server]$ service dhcpd restart

At this point you should be able to at least attempt a PXE Boot but be served with an error stating can not boot from kernel (or something of the similar).

Setting up Samba and the files to use to install Windows 7

Some assumptions here:

1. You have a working ISO for Windows 7 that you have already tested and installed from
2. You have a capable method/way of transferring this ISO to your PXE server (if you’re on Windows, look at winscp — if you’re on Linux, man scp )
3. We are using the /dist directory as our files distribution directory

First you need to copy the ISO. Assuming we are running on Linux and the file is located at /root/windows7.iso, lets execute an SCP. 192.168.1.102 is the IP of my server but make sure to change it to yours

[your_desktop] $ scp /root/windows7.iso root@192.168.1.102:/root/

Now that the file is there, lets make sure we’re back operating on the PXE server. From the PXE server, execute the following:

[pxe_server] $ mkdir /mnt/iso
[pxe_server]$ mount -t iso9660 -o loop /root/windows7.iso /mnt/iso

Now we have the ISO mounted. We must copy the files from here to our “distribution” directory which will be used to house the files for distribution through Samba. Samba is required because we need a way to serve the files to Windows — Samba is how you do this.

The Windows 7 installer I have is architecture independent and allows me to pick what architecture I want to install– therefore, my directory hierarchy is configured as such

[pxe_server]$ mkdir /dist
[pxe_server]$ mkdir /dist/OS
[pxe_server]$ mkdir /dist/Windows
[pxe_server]$ mkdir /dist/Windows/7
[pxe_server]$ cp -Rv /mnt/iso/* /dist/Windows/7/
[pxe_server]$ umount /mnt/iso

Now we have all of the files stored locally on our server and it is time to setup Samba. I had to install this as well:

[pxe_server]$ yum install samba

I made some changes to the configuration file (/etc/samba/smb.conf) that I will note below. I will also supply a picture of my file with all of the commented lines removed so you can see what it should look like in its entirety

This will be displayed as:

ORIGINAL_SETTING => MODIFIED_SETTING

If there was a comment in front of (like the interfaces line below) then I showed the modified value to remove the comment

security = user                                 =>               security = share
# interfaces = lo eth0 [...]               =>                  interfaces = lo eth0
# netbios name = MYSERVER         =>                  netbios name = MYSERVER
passdb backend = tdbsam             =>                 #       passdb backend = tdbsam

I also appended the following lines. These following lines will make our /dist/OS/Windows/7 directory mountable later on

[Windows7PXE]
	comment 	=	Windows 7 PXE DVD Install Files
	path		=	/dist/OS/Windows/7
	only guest	=	yes
	guest ok	=	yes
	writable	=	no
	guest account	=	nobody
	public		=	yes

And a picture of my file to show you how my configuration looks

As you can see, I left most of the default configuration in because at this time I am not very concerned with it.

We use security = share because we want our share to handle the security parameters where we are telling it to allow guests. You may run into other problems/extra configuration steps if you wish to use any security backend or user backend

Now lets restart samba

[pxe_server]$ service samba restart

If you have that extra Windows PC laying around, go ahead and try to browse to share to make sure it is available.

192.168.1.102 is my IP so make sure you change that!

C:\ >net view \\192.168.1.102\

You should see something of the similar

Creating a WinPE disk image to boot (both AMD64 & x86)

Here comes the somewhat tricky part — I guess its most of an annoying part than it is a tricky part. I could probably supply the ISO’s but Microsoft may get mad (eh?) about that so I will do my best to document this. If anyone needs the ISO’s that I was able to boot with, please email me or message me and I will be happy to get them to you.

1. First, install the Windows Automated Installation Pack — this can be found from: The Windows Download Site — You will need to verify with Microsoft After Clicking This — the bad part about this is it downloads an IMG file which will need to be burned to a disk and installed. I already had this installed on my system from a previous time so I did not have to do this step
2. Once you have these installed navigate the following

Start -> All Programs -> Microsoft Windows AIK -> "Deployment Tools Command Prompt"

This is basically a shell that is just a command prompt with you automatically placed into the necessary directory to use all the tools you need to make a WinPE image and a bootable ISO
3. Once here, run the following commands in order. ~~First I will cover making an x86 (32-bit) bootable ISO and then an AMD64 (x86_64 / 64-bit) bootable ISO~~

x86

C:\Program Files\Windows AIK\Tools\PETools >   copype.cmd x86 c:\winpe_x86
C:\Program Files\Windows AIK\Tools\PETools >   oscdimg -n -bc:\winpe_x86\etfsboot.com c:\winpe_x86\ISO c:\winpe_x86\winpe_x86.iso

Critical Step: This threw me off because Windows Technet documentation isnt all that great but you need to open your destination (c:\winpe_x86) and copy the winpe.wim file into ISO/sources/ and rename it to boot.wim and regenerate the ISO. More specifically do this

1. Open c:\winpe_x86
2. Select the winpe.wim file and type ctrl+c or right click and select “copy”
3. Double click ISO folder
4. Double click sources folder
5. Type ctrl+v or right click and select “paste”
6. Right click the newly pasted “winpe.wim” and rename to “boot.wim”
7. Execute (again): oscdimg -n -bc:\winpe_x86\etfsboot.com c:\winpe_x86\ISO c:\winpe_x86\winpe_x86.iso

Copy your newly created c:\winpe_x86\winpe_x86.iso to your PXEBoot server — preferably use WinSCP to place it into the root directory (I will use the assumption you copied it there later)

~~AMD64~~

C:\Program Files\Windows AIK\Tools\PETools >  copype.cmd amd64 c:\winpe_amd64
C:\Program Files\Windows AIK\Tools\PETools >  oscdimg -n -bc:\winpe_amd64\etfsboot.com c:\winpe_amd64\ISO c:\winpe_amd64\winpe_amd64.iso

Critical Step: This threw me off because Windows Technet documentation isnt all that great but you need to open your destination (c:\winpe_amd64) and copy the winpe.wim file into ISO/sources/ and rename it to boot.wim and regenerate the ISO. More specifically do this

1. Open c:\winpe_amd64
2. Select the winpe.wim file and type ctrl+c or right click and select “copy”
3. Double click ISO folder
4. Double click sources folder
5. Type ctrl+v or right click and select “paste”
6. Right click the newly pasted “winpe.wim” and rename to “boot.wim”
7. Execute (again): oscdimg -n -bc:\winpe_amd64\etfsboot.com c:\winpe_amd64\ISO c:\winpe_amd64\winpe_amd64.iso

Copy your newly created c:\winpe_amd64\winpe_amd64.iso to your PXEBoot server — preferably use WinSCP to place it into the root directory (/root) (I will use the assumption you copied it there later)

This image (winpe_x86.iso) are bootable Windows Preboot Environment images. These will bring you to nothing but a command window with the minimal services and network configuration running.

Achieving the Boot Process

Now, as previously stated, you should be able to PXE boot but be left with no graphical menu or the ability to really do anything. This is where that changes.

We need to create the menu file that the pxelinux.0 boot process can load to display a graphical menu to you. As I said above, you will have to create this inside the /tftpboot/pxelinux.cfg directory with the name of default . PXELinux requires the file be named exactly this in that directory.

There is not much I can do here but show you my menu (Picture updated 20110606)

A break down of the menu entries and what they mean are below:

# This says to load the menu.c32 file (as I noted above).  This can be found in /tftpboot -- if you change this, it must be relative of /tftpboot
default menu.c32
 
# Your title
MENU TITLE PXE Network Installation (new)
 
# Your timeout in seconds (though for some reason mine no longer counts down)
timeout 300
 
# See above explanations
Label Windows 7 (Attended)
 
# kernel is the "kernel" or the disk image we are going to load.  In this case, we are loading our updated/upgraded syslinux memdisk image
        KERNEL images/Windows_7/x86/memdisk
 
# APPEND is what we are appending to the "kernel" parameters of memdisk.  This says we are using an ISO (to load the ISO module)
# and our RAM disk is our AMD64 WinPE bootable ISO image
# The real path for this is /tftpboot/images/Windows_7/x86/
        APPEND iso initrd=images/Windows_7/x86/winpe_x86.iso

Now, we must copy the image files from the /root directory into the above mentioned paths relative to that of /tftpboot. So, execute the following:

[pxe_server]$ cp /root/winpe_x86.iso /tftpboot/images/Windows_7/x86/winpe_x86.iso
[pxe_server]$ cp /root/winpe_amd64.iso /tftpboot/images/Windows_7/x86_64/winpe_amd64.iso

Now, you should be able to PXEBoot successfully

If all goes well you should see a screen that looks like the following

Selecting the 32-bit one, you should see a successful boot beginning like this

Then you should be prompted with the “Press any key to boot from CD”. You should only be prompted with this if you have a drive with data/OS on it already

How to mount Samba share when in WinPE and Install Windows 7

Assuming you’ve made it this far, you’re doing great! If not, please don’t hesitate to ask any questions!

Now you are inside of WinPE.

Now, remember your IP of your PXE server and the share name we setup with Samba? Well, now we need to mount it by using the “net use” command. One silly part on this is that you must supply a username and password even if its bogus! If you don’t you will be prompted with an error saying the “server service” is not started– that service is used to browse networks/computers and is somewhat useless in the eyes of WinPE. But, since we’re not browsing and we know where we want to go, execute the following:

Note that my IP is 192.168.1.102 — change this to yours

net use k: \\192.168.1.102\Windows7PXE anon /user:192.168.1.102\anon

If all goes well you should see a message that says the command executed successfully and then be able to change directories to K:\ by simply typing

k:

As displayed below

Now, execute setup.exe by simply typing

K:\ > setup.exe

And you will be prompted with the Windows 7 installation Window.

And this point the installation is pretty generic. You will be able to partition the disk from within here and kick off the installation. After that, once the system installs you will be at a successful initial configuration screen to input a username and password.

An extra note on how to do this with Windows 7 in VirtualBox

I wanted to test this thing out in a Virtual Machine rather than getting up and down to go back into the datacenter in the office. Well, heres a couple issues

1. VirtualBox has a number of separate devices it can emulate — AMD PcNet II/III and 3 types of Intels
2. Windows 7 does not support AMD PcNet and finding drivers for it to inject into the WinPE image is near impossible
3. The Intel devices fail to boot
4. Windows 7 does have built-in support for the Intel devices

What to do?

Simple!

Setup two NIC cards — one to use the AMD PcNet III so you can PXE boot and the other for the Intel Desktop so you can get an IP from with WinPE natively!

Select the AMD PcNet Card for the Primary:

Select the Intel for the Secondary:

Boot and Press F12 to get to the boot menu and then press L then obtain the PXE Menu

And confirm you have an IP!

This tripped me up but then I slapped my forhead and went “duuurrrr” and realized I could just use two NICs!

That concludes this tutorial/document. If you have any questions please do not hesitate to ask. I will surely answer them as fast as I can!

I have used the following resources:
Initial PXE Resources for CentOS (This document is horrible): http://wiki.centos.org/HowTos/PXE/PXE_Setup
Got me started: http://www.savelono.com/linux/how-to-install-windows-7-over-a-network-using-linux-pxe-dnsmasq-and-samba.html

Sphere: Related Content

Category Categories: Computers, Home, Linux/Unix, Networks/Servers, Windows | Tag Tags: boot windows 7 from linux, centos pxeboot windows, netboot windows from linux, network boot windows 7 linux, pxe linux, pxeboot, pxelinux, samba, samba windows 7, syslinux, windows, windows 7, windows pxeboot from centos | 30 Comments

[Linux/MySQL] Monitor MySQL for Connection

February 26, 2011 | Posted by Chuck Kozler

While the title is a little vague, I want to give some background information on this.

I had to monitor MySQL for a connection and the MySQL logs werent helping me. Netstat couldnt work for me either because this was a multi-threaded application and at some point one of the many MySQL connections was not making it through and I had to be doubly sure that it wasnt even trying to connect.

This quick little command is what I used and figured someone else might find it useful

$ sudo strace -p `ps -Aef | grep -i mysqld | grep -iv grep | awk '{ print $2 }'`

Pretty self explanitory– you run strace on the MySQL PID. When a process or MySQL connection attempt is made you will see the following in the output

fcntl(12, F_SETFL, O_RDWR|O_NONBLOCK) = 0
accept(12, {sa_family=AF_FILE, NULL}, [2]) = 13
fcntl(12, F_SETFL, O_RDWR) = 0
getsockname(13, {sa_family=AF_FILE, path=”/var/run/mysql”}, [30]) = 0
fcntl(13, F_SETFL, O_RDONLY) = 0
fcntl(13, F_GETFL) = 0×2 (flags O_RDWR)
fcntl(13, F_SETFL, O_RDWR|O_NONBLOCK) = 0
setsockopt(13, SOL_IP, IP_TOS, [8], 4) = -1 EOPNOTSUPP (Operation not supported)
futex(0x7f3eb85f53a4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7f3eb85f53a0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

This covers ensuring the socket connection is being attempted and nothing more.

Sphere: Related Content

Category Categories: Computers, Linux/Unix, Networks/Servers | Tag Tags: Ensure MySQL is connected to, Linux, Monitor MySQL for Connection, MySQL, MySQL Connection, MySQL PID Connection Monitor | No Comments

Charles Kozlers Personal Website

Tutorials, scripts, and fixes for desktops and servers

Categories

Recent Posts

Archives

Spam Blocked

StatPress

[Mono/Windows/Linux/C#] C# Mono No STDOUT or STDERR To Console on Windows

[Linux/Perl] Generic Cron Backup Script v2

[Linux/Perl] Generic Cron Backup Script

[Bash/Linux] POC – Rip YouTube Playlist

[MySQL] Backup all MySQL databases

[FreeBSD] Nightly ZFS Snapshot Script

[FreeBSD] FreeBSD 8.2 Auto Installation with Packages

[FreeBSD] FreeBSD 8.2 init-based Auto Installation

Link

Prerequisites & Assumptions

My Preconfigured PXE-Bootable FreeBSD AutoInstaller

What is this? – Configuration Details

Configuring PXE Linux to Boot FreeBSD & FreeBSD Auto Install

Regular FreeBSD Manual Install

My FreeBSD AutoInstall

Modifying the AutoInstall for Your Environment

[Linux] Windows 7 PXE Boot From Linux

Installing TFTP & PXELinux on CentOS 5.6

Setting up PXELinux for booting WinPE

Upgrade SysLinux memdisk Image

Configure DHCP

Setting up Samba and the files to use to install Windows 7

Creating a WinPE disk image to boot (both AMD64 & x86)

Achieving the Boot Process

How to mount Samba share when in WinPE and Install Windows 7

An extra note on how to do this with Windows 7 in VirtualBox

[Linux/MySQL] Monitor MySQL for Connection

« Older Entries

Blogroll