scp and sftp for file transfers

Copyright Phillip Farrell. Last revision November 24, 2004

Table of Contents: Getting ssh programs for your computer ssh for remote login ssh authentication methods scp and sftp for file transfers ssh to run remote commands

The ssh protocol is designed to create encrypted "channels" for communication between computers. In addition to remote shell logins, these channels can be used for other types of connections. In particular, there are standard programs for copying files between computers using ssh encrypted channels. These program are called scp and sftp on Unix systems. GUI based programs that implement the same protocols are available for Macintosh and Windows PCs.

scp is a direct replacement for the old Berkeley rcp command, and is used to copy one or more files between computers with a single command line. sftp is a direct replacement for the standard ftp program, and allows you to interactive browse a remote file system and select files for copying (in either direction) with interactive commands.

scp

scp works just like the standard local Unix copy command, cp, except that you put the name of the remote system, followed by a colon, in front of the pathname specification for the remote file.

You can copy either direction, to or from the remote system, with scp. If no directory is specified for the remote system, your home directory there is assumed.

scp will prompt you for the password on the remote computer, or not, depending upon the type of authentication system you have set up for ssh.

Just like cp, scp will silently remove any existing file that is "in the way" of making its copy, at either end. But unlike cp, there is no -i option that you can use with scp to force it to ask you first before overwriting an existing file. If you think that there is any chance that an existing file may have the same name as the one you trying to copy, you need to check first yourself.

If you are copying from a remote computer to the local computer, you can simply use ls to make sure that your desired new file name is not already taken locally. If you are copying from the local computer to a remote computer, run a remote ls with ssh in remote command mode to check if your desired name is already being used remotely, for example,

ssh remotehost ls

Examples:

• scp myfile toquima:
  Copies "myfile" to my home directory on toquima.
• scp myfile toquima:/newdir/newname
  Copies "myfile" to "/newdir/newname" on toquima.
• scp toquima:myotherfile .
  Gets a copy of "myotherfile" from my home directory on toquima and puts it in my current working directory on the local machine (designated in standard UNIX fashion by the "dot" (.) character).

Just like the cp command, scp has a -p option to propagate the permission settings of the original file to the copy (otherwise the copy is made with the normal settings for new files), and a -r option to copy an entire directory tree with one command.

scp creates a completely transparent encrypted data channel between the two machines, so binary data (such as images or executable programs) is preserved correctly. This also means that scp is unable to perform automatic end-of-line termination conversion between different types of operating systems, as can be done with ftp in "ascii" mode. That will not be a problem when copying between Unix systems, which all use the same end-of-line convention.

sftp

ftp has been a standard for file transfer since the beginning of the Internet. It gives you a special interactive login on a remote computer with a limited set of commands that let you perform file operations, such as browsing the remote file system, copying files back and forth, making directories on the remote system, and removing files from the remote system, all subject, of course, to the permissions available to your account on the remote system. The problem with ftp is that it passes your password, and all file contents, across the network in plain-text.

sftp is a direct replacement for the ftp program. It gives you the same type of interactive login with special file manipulation commands. In fact, it provides more file operations than plain ftp. Like ssh and scp, it completely encrypts your password when logging into the remote system and also encrypts all files while in transit, so your password and data are safe from spying hackers.

sftp only works with servers running ssh version 2. Pangea and all Sweet Hall workstations run version 2. If the server you want to use only runs version 1 of ssh, you must use scp instead for file transfers.

Like scp, sftp creates a completely transparent encrypted data channel between the two machines, so binary data (such as images or executable programs) is preserved correctly. Unfortunately, this means that sftp is unable to perform the automatic end-of-line termination conversion between different types of operating systems that can be done with ftp in "ascii" mode. That will not be a problem when copying between Unix systems, which all use the same end-of-line convention. When using an sftp client program on a Windows or Macintosh PC, there may be a local option to perform this end-of-line termination conversion.

Syntax:

sftp remotehostname

Start sftp by simply giving the remote computer hostname or IP address on the command line. After prompting for your remote computer password (skipped if you have setup password-less authentication), you will get a new prompt that looks like

sftp>

At this point, you can give commands to browse the remote file system and copy files back and forth. It is easiest to change the working directory on the local system to the location where you want to find or put files first, before starting sftp, although it does have special commands to move around in the local file system as well as the remote file system.

The basic file manipulation commands are the same as ftp. See the TCP standards - telnet and ftp web page for a description of those commands, or use the on-line manual entry.

REFERENCE: http://pangea.stanford.edu/computerinfo/unix/netcommands/scpsftp.html

Java Miscellaneous - 1

The differences between final and static?

A final instance field must to be initialized when the object is constructed. This must be guaranteed. And the final instance field can not be modified.

A static instance field will be shared among all instance objects of the class.


How to declare a Constant?
e.g. public static final double PI = 3.14159265358979323846;


Static Characteristic:

A static method can not access a instant field and can only access static fields.

Use the Class name to invoke static methods and static fields, instead of a Object of the class.

Miscellaneous of Video problem in Linux

The Video problem is a quite complex problem usually. Not because of it is hard to resolve. On the contrast, there are far more different solutions available in most cases for a same problem on the Internet. They usually confused most people. It often happens that after the problem is solved when as much as the solutions have been tried, we even don't know which steps make it happens.

So I really need to figure out my own problem solving procedure.

Firstly, Check the Log of the X window.
That should be in the file of "/var/log/Xorg.0.log".
Something should be useful there.

Dynamically Allocating Multidimensional Arrays

Dynamically Allocating Multidimensional Arrays

We've seen that it's straightforward to call malloc to allocate a block of memory which can simulate an array, but with a size which we get to pick at run-time. Can we do the same sort of thing to simulate multidimensional arrays? We can, but we'll end up using pointers to pointers.

If we don't know how many columns the array will have, we'll clearly allocate memory for each row (as many columns wide as we like) by calling malloc, and each row will therefore be represented by a pointer. How will we keep track of those pointers? There are, after all, many of them, one for each row. So we want to simulate an array of pointers, but we don't know how many rows there will be, either, so we'll have to simulate that array (of pointers) with another pointer, and this will be a pointer to a pointer.

This is best illustrated with an example:

#include

int **array;
array = malloc(nrows * sizeof(int *));
if(array == NULL)
{
fprintf(stderr, "out of memory\n");
exit or return
}
for(i = 0; i < nrows; i++) { array[i] = malloc(ncolumns * sizeof(int)); if(array[i] == NULL) { fprintf(stderr, "out of memory\n"); exit or return } }

array is a pointer-to-pointer-to-int: at the first level, it points to a block of pointers, one for each row. That first-level pointer is the first one we allocate; it has nrows elements, with each element big enough to hold a pointer-to-int, or int *. If we successfully allocate it, we then fill in the pointers (all nrows of them) with a pointer (also obtained from malloc) to ncolumns number of ints, the storage for that row of the array. If this isn't quite making sense, a picture should make everything clear:



Once we've done this, we can (just as for the one-dimensional case) use array-like syntax to access our simulated multidimensional array. If we write

array[i][j]

we're asking for the i'th pointer pointed to by array, and then for the j'th int pointed to by that inner pointer. (This is a pretty nice result: although some completely different machinery, involving two levels of pointer dereferencing, is going on behind the scenes, the simulated, dynamically-allocated two-dimensional ``array'' can still be accessed just as if it were an array of arrays, i.e. with the same pair of bracketed subscripts.)

If a program uses simulated, dynamically allocated multidimensional arrays, it becomes possible to write ``heterogeneous'' functions which don't have to know (at compile time) how big the ``arrays'' are. In other words, one function can operate on ``arrays'' of various sizes and shapes. The function will look something like

func2(int **array, int nrows, int ncolumns)
{
}

This function does accept a pointer-to-pointer-to-int, on the assumption that we'll only be calling it with simulated, dynamically allocated multidimensional arrays. (We must not call this function on arrays like the ``true'' multidimensional array a2 of the previous sections). The function also accepts the dimensions of the arrays as parameters, so that it will know how many ``rows'' and ``columns'' there are, so that it can iterate over them correctly. Here is a function which zeros out a pointer-to-pointer, two-dimensional ``array'':

void zeroit(int **array, int nrows, int ncolumns)
{
int i, j;
for(i = 0; i < nrows; i++) { for(j = 0; j < ncolumns; j++) array[i][j] = 0; } } Finally, when it comes time to free one of these dynamically allocated multidimensional ``arrays,'' we must remember to free each of the chunks of memory that we've allocated. (Just freeing the top-level pointer, array, wouldn't cut it; if we did, all the second-level pointers would be lost but not freed, and would waste memory.) Here's what the code might look like: for(i = 0; i < nrows; i++) free(array[i]); free(array); Read sequentially: prev next up top This page by Steve Summit // Copyright 1996-1999 // mail feedback

Construct Dynamic Array

In fact the title is not fairly accurate in the context since the Dynamic Array is not supported in C programming officially. Although some others language provide this latitude, C requires that the Array size have to be assign during the compile time instead of run time. In this case, the use of variable as the index of an Array such as: int x; int array[x]; is prohibited. The advantages of this sort of array usage are that the stack would not be complicated, the function call would not be expensive and the program would run faster.
However that doesn't mean that the Dynamic Array is impossible in C. The Dynamic Memory could be used to implement this function:

void main()
{
int *ptr = NULL;
int arr_size;
printf("Enter the size of array : ");
scanf("%d",&arr_size);
ptr = (int*) malloc(arr_size*(sizeof(int));
}

After this you can use ptr[0],ptr[1],.....ptr[n] as an array.

References: GeekInterview.com C programming interview questions 69 of 437

The Bash Shell Startup Files

The shell program /bin/bash (hereafter referred to as just "the shell") uses a collection of startup files to help create an environment. Each file has a specific use and may affect login and interactive environments differently. The files in the /etc directory generally provide global settings. If an equivalent file exists in your home directory it may override the global settings.

An interactive login shell is started after a successful login, using /bin/login, by reading the /etc/passwd file. This shell invocation normally reads /etc/profile and its private equivalent ~/.bash_profile upon startup.

An interactive non-login shell is normally started at the command-line using a shell program (e.g.,[prompt]$/bin/bash) or by the /bin/su command. An interactive non-login shell is also started with a terminal program such as xterm or konsole from within a graphical environment. This type of shell invocation normally copies the parent environment and then reads the user's ~/.bashrc file for additional startup configuration instructions.

A non-interactive shell is usually present when a shell script is running. It is non-interactive because it is processing a script and not waiting for user input between commands. For these shell invocations, only the environment inherited from the parent shell is used.

The file ~/.bash_logout is not used for an invocation of the shell. It is read and executed when a user exits from an interactive login shell.

Many distributions use /etc/bashrc for system wide initialization of non-login shells. This file is usually called from the user's ~/.bashrc file and is not built directly into bash itself. This convention is followed in this section.

For more information see info bash -- Nodes: Bash Startup Files and Interactive Shells.

 

Note

Most of the instructions below are used to create files located in the /etc directory structure which requires you to execute the commands as the root user. If you elect to create the files in user's home directories instead, you should run the commands as an unprivileged user.

/etc/profile

Here is a base /etc/profile. This file starts by setting up some helper functions and some basic parameters. It specifies some bash history parameters and, for security purposes, disables keeping a permanent history file for theroot user. It also sets a default user prompt. It then calls small, single purpose scripts in the /etc/profile.ddirectory to provide most of the initialization.

For more information on the escape sequences you can use for your prompt (i.e., the PS1 environment variable) see info bash -- Node: Printing a Prompt.

cat > /etc/profile << "EOF" # Begin /etc/profile # Written for Beyond Linux From Scratch # by James Robertson  # modifications by Dagmar d'Surreal   # System wide environment variables and startup programs.  # System wide aliases and functions should go in /etc/bashrc.  Personal # environment variables and startup programs should go into # ~/.bash_profile.  Personal aliases and functions should go into # ~/.bashrc.  # Functions to help us manage paths.  Second argument is the name of the # path variable to be modified (default: PATH) pathremove () {         local IFS=':'         local NEWPATH         local DIR         local PATHVARIABLE=${2:-PATH}         for DIR in ${!PATHVARIABLE} ; do                 if [ "$DIR" != "$1" ] ; then                   NEWPATH=${NEWPATH:+$NEWPATH:}$DIR                 fi         done         export $PATHVARIABLE="$NEWPATH" }  pathprepend () {         pathremove $1 $2         local PATHVARIABLE=${2:-PATH}         export $PATHVARIABLE="$1${!PATHVARIABLE:+:${!PATHVARIABLE}}" }  pathappend () {         pathremove $1 $2         local PATHVARIABLE=${2:-PATH}         export $PATHVARIABLE="${!PATHVARIABLE:+${!PATHVARIABLE}:}$1" }   # Set the initial path export PATH=/bin:/usr/bin  if [ $EUID -eq 0 ] ; then         pathappend /sbin:/usr/sbin         unset HISTFILE fi  # Setup some environment variables. export HISTSIZE=1000 export HISTIGNORE="&:[bf]g:exit"  # Setup a red prompt for root and a green one for users. NORMAL="\[\e[0m\]" RED="\[\e[1;31m\]" GREEN="\[\e[1;32m\]" if [[ $EUID == 0 ]] ; then   PS1="$RED\u [ $NORMAL\w$RED ]# $NORMAL" else   PS1="$GREEN\u [ $NORMAL\w$GREEN ]\$ $NORMAL" fi  for script in /etc/profile.d/*.sh ; do         if [ -r $script ] ; then                 . $script         fi done  # Now to clean up unset pathremove pathprepend pathappend  # End /etc/profile EOF

The /etc/profile.d Directory

Now create the /etc/profile.d directory, where the individual initialization scripts are placed:

install --directory --mode=0755 --owner=root --group=root /etc/profile.d

/etc/profile.d/dircolors.sh

This script uses the ~/.dircolors and /etc/dircolors files to control the colors of file names in a directory listing. They control colorized output of things like ls --color. The explanation of how to initialize these files is at the end of this section.

cat > /etc/profile.d/dircolors.sh << "EOF" # Setup for /bin/ls to support color, the alias is in /etc/bashrc. if [ -f "/etc/dircolors" ] ; then         eval $(dircolors -b /etc/dircolors)          if [ -f "$HOME/.dircolors" ] ; then                 eval $(dircolors -b $HOME/.dircolors)         fi fi alias ls='ls --color=auto' EOF

/etc/profile.d/extrapaths.sh

This script adds several useful paths to the PATH and PKG_CONFIG_PATH environment variables. If you want, you can uncomment the last section to put a dot at the end of your path. This will allow executables in the current working directory to be executed without specifiying a ./, however you are warned that this is generally considered a security hazard.

cat > /etc/profile.d/extrapaths.sh << "EOF" if [ -d /usr/local/lib/pkgconfig ] ; then         pathappend /usr/local/lib/pkgconfig PKG_CONFIG_PATH fi if [ -d /usr/local/bin ]; then         pathprepend /usr/local/bin fi if [ -d /usr/local/sbin -a $EUID -eq 0 ]; then         pathprepend /usr/local/sbin fi  if [ -d ~/bin ]; then         pathprepend ~/bin fi #if [ $EUID -gt 99 ]; then #        pathappend . #fi EOF

/etc/profile.d/readline.sh

This script sets up the default inputrc configuration file. If the user does not have individual settings, it uses the global file.

cat > /etc/profile.d/readline.sh << "EOF" # Setup the INPUTRC environment variable. if [ -z "$INPUTRC" -a ! -f "$HOME/.inputrc" ] ; then         INPUTRC=/etc/inputrc fi export INPUTRC EOF

/etc/profile.d/umask.sh

Setting the umask value is important for security. Here the default group write permissions are turned off for system users and when the user name and group name are not the same.

cat > /etc/profile.d/umask.sh << "EOF" # By default we want the umask to get set. if [ "$(id -gn)" = "$(id -un)" -a $EUID -gt 99 ] ; then   umask 002 else   umask 022 fi EOF

/etc/profile.d/X.sh

If X is installed, the PATH and PKG_CONFIG_PATH variables are also updated.

cat > /etc/profile.d/X.sh << "EOF" if [ -x /usr/X11R6/bin/X ]; then         pathappend /usr/X11R6/bin fi if [ -d /usr/X11R6/lib/pkgconfig ] ; then         pathappend /usr/X11R6/lib/pkgconfig PKG_CONFIG_PATH fi EOF

/etc/profile.d/i18n.sh

This script sets an environment variable necessary for native language support. A full discussion on determining this variable can be found on the LFS Bash Shell Startup Files page.

cat > /etc/profile.d/i18n.sh << "EOF" # Set up i18n variables export LANG=_.<@modifiers> EOF

Other Initialization Values

Other initialization can easily be added to the profile by adding additional scripts to the /etc/profile.d directory.

/etc/bashrc

Here is a base /etc/bashrc. Comments in the file should explain everything you need.

cat > /etc/bashrc << "EOF" # Begin /etc/bashrc # Written for Beyond Linux From Scratch # by James Robertson  # updated by Bruce Dubbs   # System wide aliases and functions.  # System wide environment variables and startup programs should go into # /etc/profile.  Personal environment variables and startup programs # should go into ~/.bash_profile.  Personal aliases and functions should # go into ~/.bashrc  # Provides a colored /bin/ls command.  Used in conjunction with code in # /etc/profile.  alias ls='ls --color=auto'  # Provides prompt for non-login shells, specifically shells started # in the X environment. [Review the LFS archive thread titled # PS1 Environment Variable for a great case study behind this script # addendum.]  NORMAL="\[\e[0m\]" RED="\[\e[1;31m\]" GREEN="\[\e[1;32m\]" if [[ $EUID == 0 ]] ; then   PS1="$RED\u [ $NORMAL\w$RED ]# $NORMAL" else   PS1="$GREEN\u [ $NORMAL\w$GREEN ]\$ $NORMAL" fi  # End /etc/bashrc EOF

~/.bash_profile

Here is a base ~/.bash_profile. If you want each new user to have this file automatically, just change the output of the command to /etc/skel/.bash_profile and check the permissions after the command is run. You can then copy/etc/skel/.bash_profile to the home directories of already existing users, including root, and set the owner and group appropriately.

cat > ~/.bash_profile << "EOF" # Begin ~/.bash_profile # Written for Beyond Linux From Scratch # by James Robertson  # updated by Bruce Dubbs   # Personal environment variables and startup programs.  # Personal aliases and functions should go in ~/.bashrc.  System wide # environment variables and startup programs are in /etc/profile. # System wide aliases and functions are in /etc/bashrc.  append () {   # First remove the directory   local IFS=':'   local NEWPATH   for DIR in $PATH; do      if [ "$DIR" != "$1" ]; then        NEWPATH=${NEWPATH:+$NEWPATH:}$DIR      fi   done    # Then append the directory   export PATH=$NEWPATH:$1 }  if [ -f "$HOME/.bashrc" ] ; then   source $HOME/.bashrc fi  if [ -d "$HOME/bin" ] ; then   append $HOME/bin fi  unset append  # End ~/.bash_profile EOF

~/.bashrc

Here is a base ~/.bashrc. The comments and instructions for using /etc/skel for .bash_profile above also apply here. Only the target file names are different.

cat > ~/.bashrc << "EOF" # Begin ~/.bashrc # Written for Beyond Linux From Scratch # by James Robertson   # Personal aliases and functions.  # Personal environment variables and startup programs should go in # ~/.bash_profile.  System wide environment variables and startup # programs are in /etc/profile.  System wide aliases and functions are # in /etc/bashrc.  if [ -f "/etc/bashrc" ] ; then   source /etc/bashrc fi  # End ~/.bashrc EOF

~/.bash_logout

This is an empty ~/.bash_logout that can be used as a template. You will notice that the base ~/.bash_logout does not include a clear command. This is because the clear is handled in the /etc/issue file.

cat > ~/.bash_logout << "EOF" # Begin ~/.bash_logout # Written for Beyond Linux From Scratch # by James Robertson   # Personal items to perform on logout.  # End ~/.bash_logout EOF

/etc/dircolors

If you want to use the dircolors capability, then run the following command. The /etc/skel setup steps shown above also can be used here to provide a ~/.dircolors file when a new user is set up. As before, just change the output file name on the following command and assure the permissions, owner, and group are correct on the files created and/or copied.

dircolors -p > /etc/dircolors

If you wish to customize the colors used for different file types, you can edit the /etc/dircolors file. The instructions for setting the colors are embedded in the file.

Finally, Ian Macdonald has written an excellent collection of tips and tricks to enhance your shell environment. You can read it online at http://www.caliban.org/bash/index.shtml.

Last updated on 2008-05-09 08:00:42 -0500

Reference: Directly Copy From  http://www.linuxfromscratch.org/blfs/view/svn/postlfs/profile.html

The Usage of rc.conf file

Under the directory /etc, the rc.conf file is definitely regarded as the system configuration file. During the first attempting installation of FreeBSD, the utilization of rc.conf is of following:

1. To run the Gnome Display Manager (GDM) in order to start up the GNOME environment as long as the system start up. Adding gdm_enable="YES"
   
2. To Customize the DHCP of the network. Adding ifconfig_em0="DHCP"