A Guide to the sed Stream Editor

Function Overview:
sed is a stream editor that reads text from files or input streams line by line, edits the text according to user-specified patterns or commands, and then outputs the result to the screen or a file. When used in conjunction with regular expressions, it is incredibly powerful.

Syntax:

sed [options] 'command' file(s)
sed [options] -f scriptfile file(s)

Explanation:
sed first stores each line of the text in a temporary buffer called the “pattern space.” It then processes the content of this buffer according to the given sed commands. Once the processing is complete, the result is output to the terminal, and sed moves on to the next line. The content of the file itself is not altered unless the -i option is used. sed is mainly used to edit one or more text files, simplify repeated text file operations, or create text transformation scripts. Its functionality is similar to awk, but sed is simpler and less capable of handling column-specific operations, while awk is more powerful in that regard.

Options:

  • -e: Use the specified commands to process the input text file.
  • -n: Suppress automatic output (only prints lines modified when used with the p command).
  • -h: Display help information.
  • -V: Display version information.

Parameters:

  • command: The command to be executed.
  • file(s): One or more text files to be processed.
  • scriptfile: A file containing a list of commands to execute.

Common Actions:

  • a: Append text after the current line.
  • i: Insert text before the current line.
  • c: Replace the selected lines with new text.
  • d: Delete the selected lines.
  • D: Delete the first line of the pattern block.
  • s: Replace specified characters.
  • h: Copy the pattern block’s content to an internal buffer.
  • H: Append the pattern block’s content to the internal buffer.
  • g: Retrieve content from the internal buffer and replace the text in the current pattern block.
  • G: Retrieve content from the internal buffer and append it to the current pattern block.
  • l: List non-printable characters in the text.
  • L: Similar to l, but specifically for handling non-ASCII characters.
  • n: Read the next input line and apply the next command to it instead of reapplying the first command.
  • N: Append the next input line to the current pattern block and insert a new line between them, changing the current line number.
  • p: Print the matching lines.
  • P: Print the first line of the pattern block.
  • q: Quit sed.
  • b label: Branch to the location marked by label in the script; if the label doesn’t exist, the branch goes to the end of the script.
  • r file: Read lines from a file.
  • t label: Conditional branch to a marked location, starting from the last line. If the condition is met, or a T/t command is used, the branch jumps to the specified label or the end of the script.
  • T label: Error branch. If an error occurs, this branches to the labeled command or the end of the script.
  • w file: Write the processed block of the pattern space to the end of a file.
  • W file: Write the first line of the pattern space to the end of a file.
  • !: Execute the following commands on all lines not selected by the current pattern.
  • =: Print the current line number.
  • #: Extend comments to the next newline character.

Replacement Commands:

  • g: Global replacement within a line (used with the s command).
  • p: Print the line.
  • w: Write the line to a file.
  • x: Exchange the text in the pattern block with the text in the internal buffer.
  • y: Translate one character to another (not used with regular expressions).
  • &: Reference to the matched string.

Basic Regular Expression (BRE) Syntax in sed:

  • ^: Match the beginning of a line.
  • $: Match the end of a line.
  • .: Match any single character except a newline.
  • *: Match zero or more of the preceding characters.
  • []: Match a single character from a specified range.
  • [^]: Match a single character not in the specified range.
  • (..): Capture a substring.
  • &: Save the matched text for later use in replacements.
  • <: Match the start of a word.
  • >: Match the end of a word.
  • x{m}: Match exactly m occurrences of x.
  • x{m,}: Match at least m occurrences of x.
  • x{m,n}: Match between m and n occurrences of x.

To match the start of a word, use \<. To match the end of a word, use \>.

Extended Regular Expression (ERE) Syntax in sed:

  • \b: Match a word boundary (not supported by default in sed regular expressions).
  • +: Match one or more occurrences of the preceding character.

Practical Examples:

1 Print specific lines:
To print only lines 1 and the last line:

sed -n '1p;$p' test.txt

2 Delete lines:
To delete the second line:

sed '2d' filename

3 Basic match and replace:
Replace spaces with hyphens:

echo "hello world" | sed 's/ /-/g'

4 Advanced match and replace:
Reverse words in a string:

echo "abc def ghi" | sed 's/\([a-zA-Z]*\) \([a-zA-Z]*\) \([a-zA-Z]*\)/\3 \2 \1/'

5 Multiple edits:
Replace “Hello” with “Hi” and “Goodbye” with “Farewell” in one command:

sed 's/Hello/Hi/; s/Goodbye/Farewell/' example.txt

6 Read a file:
Insert content from an external file after lines matching a pattern:

sed '/Line 2/r extra.txt' data.txt

7 Write to a file:
Save processed content into a new file:

sed 's/World/Everyone/' input.txt > output.txt

In summary, sed is a versatile and efficient tool for editing text in a stream, offering powerful pattern matching and text transformation capabilities when combined with regular expressions. From basic line printing to advanced text manipulation, sed serves a wide range of text processing needs.

sed流编辑器

功能说明:sed是一种流编辑器,能够从文件或输入流中逐行读取文本,并根据用户指定的模式或命令对文本进行编辑,之后将结果输出到屏幕或文件中。配合正则表达式使用功能强大。

语  法:

sed [options] ‘command’ file(s)

sed [options] -f scriptfile file(s)

补充说明:sed先把当前处理的一行文本存储在临时缓冲区中,称为“模式空间”,接着用sed命令处理缓冲区的内容,完成后输出到终端,接着处理下一行文本。文件内容并没有被改变,除非使用-i选项。sed主要用来编辑一个或多个文本文件,简化对文本文件的反复操作或者用来编写文本转换程序等。sed功能同awk类似,差别在于sed更加简单,对列处理的功能要差一些,awk功能复杂,对列处理的功能比较强大。

   项:

-e    以指定的指令来处理输入的文本文件

-n    取消默认输出(如果和p命令同时使用只会打印发生改变的行)

-h    显示帮助信息

-V   显示版本信息

参  数:

command     命令

file(s)           一个或多个文本文件

scriptfile       存放了命令的脚本文件

   作:

a     在当前行下面插入文本

i      在当前行上面插入文本

c     把选定的行改为新的文本

d     删除选择的行

D    删除模板块的第一行

s      替换指定字符

h     拷贝模板块的内容到内存中的缓冲区

H    追加模板块的内容到内存中的缓冲区

g     获得内存缓冲区的内容,并替代当前模板块中的文本

G    获得内存缓冲区的内容,并追加到当前模板块文本的后面

l      列出不能打印字符的清单

L  列出不能打印字符的清单,该选项用于非ASCII字符

n     读取下一个输入行,用下一个命令处理新的行而不是用第一个命令

N    追加下一个输入行到模板块后面并在二者间嵌入一个新行,改变当前行号码

p     打印匹配的行

P     打印模板的第一行

q     退出sed

b     lable 分支到脚本中带有标记的地方,如果分支不存在则分支到脚本的末尾

r      file 从文件中读行

t      label if分支,从最后一行开始,条件一旦满足或者T,t命令,将导致分支到带有标号的命令处,或者到脚本的末尾

T     label 错误分支,从最后一行开始,一旦发生错误或者T,t命令,将导致分支到带有标号的命令处,或者到脚本的末尾

w    file 写并追加模板块到文件的末尾

W   file 写并追加模板块的第一行到file末尾

!      表示后面的命令对所有没有被选定的行发生作用

=     打印当前行号码

#     把注释扩展到下一个换行符以前

替换命令:

g     表示行内全面替换(全局替换配合s命令使用)

p     表示打印行

w    表示把行写入一个文件

x     表示互换模板块中的文本和缓冲区中的文本

y     表示把一个字符翻译为另外的字符(但是不用于正则表达式)

1     子串匹配的标记 

&    已匹配字符串的标记

sed的基本正则表达式(BREBasic Regular Expression)语法:

^     匹配行开始

$     匹配行结束

.      匹配一个非换行符的任意字符

*     匹配0个或多个字符

[]    匹配指定范围内的一个字符

[^]   匹配不在指定范围内的一个字符

(..)  匹配子串

&    保存搜索字符用来替换其他字符

<     匹配单词的开始

>     匹配单词的结束

x{m}     重复字符x,m次

x{m,}    重复字符x,至少m次

x{m,n}  重复字符x,至少m次,不多于n次

使用 \< 来匹配单词开头,\> 来匹配单词结尾

sed的扩展正则表达式(EREExtended Regular Expression)语法:

\b    匹配单词边界,但默认的sed正则表达式语法不支持 \b

+     匹配一个或多个字符

    例:

1 打印输出

只输出指定行号的行:

$ cat test.txt

abcd 12345

b

c

d

e

输出第1行和最后一行:

$ sed -n ‘1p;$p’ test.txt

abcd 12345

e

输出第2行和第3行:

$ sed -n ‘2p;3p’ test.txt

b

c

输出第2行、第3行和第4行:

$ sed -n ‘2p;3p;4p’ test.txt

b

c

d

其中-n选项取消默认输出,p命令只打印输出指定行号的行。

只输出奇数行号的行:

$ sed -n ‘p;n’ test.txt

abcd 12345

c

e

只输出偶数行号的行:

$ sed -n ‘n;p’ test.txt

b

d

从第1行开始隔行输出:

$ sed -n ‘1~2p’ test.txt

abcd 12345

c

e

从第2行开始隔行输出:

$ sed -n ‘2~2p’ test.txt

b

d

打印匹配字符串行的下一行:

$ sed -n ‘/^b/{n;p}’ test.txt

c

$ awk ‘/^b/{getline; print}’ test.txt

c

使用l 和 L 动作打印输出行内容,并以不同的方式显示控制字符(如不可打印字符、换行符等):

  • l 动作:显示行内容,并将非打印字符(如制表符、换行符)以可视化符号显示,适用于处理 ASCII 文本。
  • L 动作:类似于 l,但专为处理多字节字符(如 UTF-8)设计,适合包含国际化字符的文本。

示例1

$ cat test.txt

ab

c     d 12345

b

c

d

e

执行 l 动作后,sed 会将每一行的内容打印出来,并将非打印字符(如换行、制表符等)显示为可视符号:

$ sed -n ‘l’ test.txt

ab$

c\td 12345$

b$

c$

d$

e$

其中

  • \t 表示制表符,\n 表示换行符。
  • $ 表示行的结尾,通常被 sed 用来可视化显示每行的结束。

示例2

$ cat test1.txt

Hello 世界

与 l 动作不同,L 更适用于处理多字节字符,特别是在显示非 ASCII 字符时:

$ sed -n ‘L’ test1.txt

Hello 世$

界$

其中多字节字符(如中文字符“世界”)会被正确显示为两行,其中 世 和 界 分别占用一行,这在某些编辑场景下可能是期望的效果。

注意,低版本sed不支持L选项!

2 删除

删除空行:

sed ‘/^$/d’ filename

删除第二行:

sed ‘2d’ filename

删除第二直到未尾所有行:

sed ‘2, $d’ filename

删除最后一行:

sed ‘$d’ filename

删除以test开头行:

sed ‘/^test/’d filename

3 简单的匹配和替换

echo “hello world” |sed ‘s/ /-/g’

hello-world 

从第一个空格开始把空格符号全局替换成’-‘符号,只不过”hello world”文本中只有一个空格。

匹配一个完整的单词并替换:

$ echo “hello world” | sed ‘s/[a-zA-Z0-9_][a-zA-Z0-9_]*/replacement/g’

replacement replacement

其中

  • [a-zA-Z0-9_] 匹配一个字母、数字或下划线。
  • [a-zA-Z0-9_]* 匹配零个或多个后续的字母、数字或下划线

在某些支持扩展正则表达式的工具(如 sed -E 或 grep -E),你可以直接使用 + 来表示一个或多个字符:

$ echo “hello world” | sed -E ‘s/[a-zA-Z0-9_]+/replacement/g’

replacement replacement

4 进阶的匹配和替换

$ echo “hello world” | sed ‘s/[a-zA-Z0-9_][a-zA-Z0-9_]*/[&]/g’

[hello] [world]

其中&表示匹配到的子串。

通过正则表达式分组和替换实现反转输出一个字符串中的空格分隔的子串:

$ echo “abc def ghi” | sed ‘s/\([a-zA-Z]*\) \([a-zA-Z]*\) \([a-zA-Z]*\)/\3 \2 \1/’

ghi def abc

如果有更多的子串,使用 sed 进行手动反转就会变得非常复杂,因为 sed 的捕获组数量有限(通常只能捕获到9个组,即 \1 到 \9)。如果需要反转更多子串,建议使用更强大的文本处理工具,如 awk 或 perl。例如:

$ echo “abc def ghi jkl mno” | awk ‘{ for (i=NF; i>0; i–) printf(“%s “, $i); print “” }’

mno jkl ghi def abc

其中

  • NF 表示字段数量,$i 表示第 i 个字段。
  • for (i=NF; i>0; i–) 从最后一个字段开始向前输出,直到第一个字段。

5 多点编辑功能

多点编辑功能可以通过 -e 选项来实现。-e 选项允许你在同一个 sed 命令中执行多个编辑操作。每个编辑命令都可以通过 -e 传递,这样你可以在一次执行中对文件或输入流进行多种编辑操作,而不需要多次调用 sed。

基本语法:

sed -e ‘command1’ -e ‘command2’ … filename

或者将多个 -e 选项合并为一个(不使用 -e 的情况下也可以):

sed ‘command1; command2’ filename

示例1 一次完成两个替换操作

$ cat example.txt

Hello World

This is a test

Goodbye World

$ sed -e ‘s/Hello/Hi/’ -e ‘s/Goodbye/Farewell/’ example.txt

Hi World

This is a test

Farewell World

你也可以不用多次使用 -e,而是通过分号分隔多个命令:

sed ‘s/Hello/Hi/; s/Goodbye/Farewell/’ example.txt

示例2 删除和替换操作的组合

假设你想要删除文件中的第 2 行,并将 “World” 替换为 “Everyone”。你可以通过以下命令来实现:

$ cat example.txt

Hello World

This is a test

Goodbye World

$ sed -e ‘2d’ -e ‘s/World/Everyone/’ example.txt

Hello Everyone

Goodbye Everyone

6 读一个文本文件

sed默认操作就是读取文本文件内容并对其进行处理。

示例1 读取并打印文件内容

$ cat input.txt

Hello World

This is a test

Goodbye World

$ sed ” input.txt

Hello World

This is a test

Goodbye World

示例2 读取并替换文件内容

假设你想将 World 替换为 Everyone,可以这样做:

$ sed ‘s/World/Everyone/’ input.txt

Hello Everyone

This is a test

Goodbye Everyone

7 使用r动作读取文件并插入内容

r 动作用于将外部文件的内容读入并插入到当前处理的文本中。指定一个文件,sed 会将该文件的内容插入到匹配的行之后。语法:

sed ‘/pattern/r file_to_read’ input_file

其中

  • /pattern/:匹配模式行(可选),即插入文件内容的位置。
  • file_to_read:你想要读取的文件。
  • input_file:原始文件,sed 将对其进行处理。

示例1

假设有一个文件 data.txt,内容如下:

Line 1

Line 2

Line 3

还有另一个文件 extra.txt,内容如下:

Extra content 1

Extra content 2

如果你想在 data.txt 的匹配 Line 2的每一行后插入 extra.txt 的内容,可以使用以下命令:

sed ‘/Line 2/r extra.txt’ data.txt

输出结果:

Line 1

Line 2

Extra content 1

Extra content 2

Line 3

8 写一个文本文件

为了将 sed 的输出保存到一个新的文件,或者覆盖现有的文件,可以使用输出重定向或 -i 选项(用于直接修改文件)。

示例1 使用输出重定向写入文件

假设你想将替换后的内容写入到一个新文件 output.txt:

$ cat input.txt

Hello World

This is a test

Goodbye World

$ sed ‘s/World/Everyone/’ input.txt > output.txt

$ cat output.txt

Hello Everyone

This is a test

Goodbye Everyone

以上将 sed 的输出结果重定向到 output.txt,不会改变原始文件 input.txt 的内容。

示例2 使用 -i 选项直接修改文件本身

如果你想直接修改 input.txt 文件本身,可以使用 -i 选项:

$ sed -i ‘s/World/Everyone/’ input.txt

$ cat input.txt

Hello Everyone

This is a test

Goodbye Everyone

示例3 在文件中添加内容

你也可以通过 sed 来插入或添加内容,并保存到文件中。假设你想在input.txt文件的第 1 行之前插入一行新文本 “ID: 1234″,并将其保存到原文件中:

$ cat input.txt

Hello Everyone

This is a test

Goodbye Everyone

$ sed -i ‘1i ID: 1234’ input.txt

$ cat input.txt

ID: 1234

Hello Everyone

This is a test

Goodbye Everyone

其中1i表示在第 1 行之前插入一行新文本。

9 使用w 动作写入文件

w 动作用于将匹配的行或处理后的内容写入到一个指定的文件。它通常用于保存处理过的内容到新的文件,而不是修改原文件。语法:

sed ‘/pattern/w output_file’ input_file

其中

  • /pattern/:匹配模式行,符合该模式的行会被写入指定的文件。
  • output_file:写入的目标文件,如果文件不存在,sed 会自动创建它。
  • input_file:原始文件,sed 将对其进行处理。

示例1

假设有一个文件 data.txt,内容如下:

Line 1

Line 2

Line 3

你想将匹配 Line 2 的行写入到文件 output.txt 中,可以使用以下命令:

sed ‘/Line 2/w output.txt’ data.txt

执行该命令后,output.txt 文件将包含以下内容:

Line 2

示例2 结合 r 和 w

假设你想读取外部文件的内容并插入到某个模式之后,同时将匹配的行写入到另一个文件中,可以这样做:

sed ‘/Line 2/r extra.txt; /Line 2/w output.txt’ data.txt

其中

  • /Line 2/r extra.txt:在匹配到 Line 2 的地方插入 extra.txt 文件的内容。
  • /Line 2/w output.txt:将匹配的 Line 2 行写入到 output.txt。

What is the od Command and How to Use It?

The od (octal dump) command is a versatile tool that outputs the contents of a specified file in various formats such as octal, decimal, hexadecimal, floating-point numbers, or ASCII characters. It displays the content to the standard output (usually the terminal), with the leftmost column showing the byte offset, starting from 0.

Function:

The od command outputs file content in various formats like octal, decimal, hexadecimal, floating-point, or ASCII, with the byte offset displayed in the leftmost column. It can handle both text and binary files and is typically used to view file data that cannot be directly displayed in the terminal, such as binary data. The command can interpret the file content and output its values in various formats, whether they are IEEE754 floating-point numbers or ASCII codes. You might also want to check out the hexdump command, which by default outputs data in hexadecimal format but isn’t as powerful as od.

Syntax:

od [OPTION…] [FILE…]

Key Options:

  • -A RADIX or --address-radix=RADIX: Specifies the radix (base) for the byte offset. By default, the offset is displayed in octal.
  • -j BYTES or --skip-bytes=BYTES: Skips the specified number of bytes before displaying the file content.
  • -N BYTES or --read-bytes=BYTES: Outputs only the specified number of bytes.
  • -S [BYTES] or --strings[=BYTES]: Outputs strings at least BYTES bytes long (default is 3).
  • -v or --output-duplicates: Ensures that duplicate data is not omitted in the output.
  • -w [BYTES] or --width[=BYTES]: Sets the number of bytes to display per line (default is 32 bytes).
  • -t TYPE or --format=TYPE: Specifies the format of the output. Options include:
    • a: Named characters (e.g., newline is shown as “nl”).
    • c: Printable characters or escaped sequences (e.g., newline is shown as “\n”).
    • d[SIZE]: Signed decimal integers of SIZE bytes (default is sizeof(int)).
    • f[SIZE]: Floating-point numbers of SIZE bytes (default is sizeof(double)).
    • o[SIZE]: Octal integers of SIZE bytes (default is sizeof(int)).
    • u[SIZE]: Unsigned decimal integers of SIZE bytes (default is sizeof(int)).
    • x[SIZE]: Hexadecimal integers of SIZE bytes (default is sizeof(int)).
    The SIZE can be specified as 1 (byte), or as uppercase letters like C (char), S (short), I (int), and L (long). For floating-point numbers, SIZE can be F (float), D (double), or L (long double).
  • --help: Displays help information.
  • --version: Displays version information.

Parameters:

  • FILE…: One or more files whose content will be displayed.

Examples:

Example 1: Basic Output

$ cat test.txt
abcd 12345
$ od test.txt 
0000000 061141 062143 030440 031462 032464 000012
0000013

In this output, the first column shows the byte offset (default in octal).

Example 2: Show Byte Offset in Decimal

$ od -Ad test.txt 
0000000 061141 062143 030440 031462 032464 000012
0000011

Example 3: Hide Byte Offset

$ od -An test.txt 
 061141 062143 030440 031462 032464 000012

Example 4: Output in Hexadecimal (4 Bytes per Group)

$ od -tx test.txt 
0000000 64636261 33323120 000a3534
0000013

Example 5: Output in Hexadecimal (1 Byte per Group)

$ od -tx1 test.txt
0000000 61 62 63 64 20 31 32 33 34 35 0a
0000013

Example 6: Display Named ASCII Characters

$ od -ta test.txt
0000000   a   b   c   d  sp   1   2   3   4   5  nl
0000013

Or display printable characters and escape sequences:

$ od -tc test.txt
0000000   a   b   c   d       1   2   3   4   5  \n
0000013

Example 7: Hexadecimal with Original Characters

$ od -tcx1 test.txt
0000000   a   b   c   d       1   2   3   4   5  \n
         61  62  63  64  20  31  32  33  34  35  0a
0000013

Example 8: Specify Bytes per Line

$ od -w8 -tc test.txt
0000000   a   b   c   d       1   2   3
0000010   4   5  \n
0000013

Example 9: Remove Spaces Between Columns

To remove spaces between columns during od output:

  1. Use -An to hide the offset.
  2. Use -v to avoid omitting duplicate data.
  3. Use -tx1 to output one byte per group in hexadecimal format, and -w1 to display one byte per line.
  4. Finally, pipe the output to awk to concatenate it into a single line.
$ od -An -w1 -tx1 test.txt | awk '{for(i=1;i<=NF;++i){printf "%s",$i}}'
616263642031323334350a

od将指定文件内容以八进制数、十进制数、十六进制数、浮点数或ASCII字符的方式输出到标准输出显示

功能说明:od将指定文件内容以八进制数、十进制数、十六进制数、浮点数或ASCII字符方式输出到标准输出显示,并且最左边一列显示字节地址偏移量,从0开始

语  法:od [OPTION…] [FILE…]

补充说明:od命令默认的显示方式是八进制数。常见的文件为文本文件和二进制文件。od命令通常用于显示或查看文件中不能直接显示在终端的字符,主要用来查看保存在二进制文件中的数据,按照指定格式解释文件中的数据并输出,不管是IEEE754格式的浮点数还是ASCII码,od命令都能按照需求输出它们的值。大家也可以了解一下hexdump命令,默认以十六进制数输出数据,但感觉hexdump命令没有od命令强大。

          项:

-A RADIX或–address-radix=RADIX        选择以何种基数表示字节地址偏移量。默认以八进制数显示

-j BYTES或–skip-bytes=BYTES               跳过指定数目的字节

-N BYTES或–read-bytes=BYTES             输出指定字节个数

-S [BYTES]或–strings[=BYTES]               输出长度不小于指定字节数的字符串,BYTES 缺省值为 3

-v或–output-duplicates                               输出时不省略重复的数据

-w [BYTES]或–width[=BYTES]                设置每行最多显示的字节个数,BYTES 缺省为 32 字节

-t TYPE或–format=TYPE                          指定输出格式,格式包括 a、c、d、f、o、u 和 x,各含义如下:

  • a:具名字符。比如换行符显示为 nl
  • c:可打印字符或反斜杠表示的转义字符。比如换行符显示为 \n
  • d[SIZE]:SIZE 字节组成一个有符号十进制整数。SIZE 缺省值为 sizeof(int)
  • f[SIZE]:SIZE 字节组成一个浮点数。SIZE 缺省为 sizeof(double)
  • o[SIZE]:SIZE 字节组成一个八进制整数。SIZE 缺省为 sizeof(int)
  • u[SIZE]:SIZE 字节组成一个无符号十进制整数。SIZE 缺省为 sizeof(int)
  • x[SIZE]:SIZE 字节组成一个十六进制整数。SIZE 缺省为 sizeof(int)

SIZE可以1数字,也可以是大写字母。如果 TYPE 是 [doux] 中的一个,那么SIZE 可以是C = sizeof(char),S = sizeof(short),I = sizeof(int),L = sizeof(long)。如果 TYPE 是 f,那么 SIZE 可以是 F = sizeof(float),D = sizeof(double) ,L = sizeof(long double)

–help           显示帮助信息

–version       显示版本信息

参  数:

FILE…         要显示内容数据的一个或多个文件

   例:

实例1
$ cat test.txt
abcd 12345
$ od test.txt 
0000000 061141 062143 030440 031462 032464 000012
0000013
输出中的第一列是字节地址偏移量,默认以八进制数显示。

实例2
设置第一列的字节偏移地址以十进制显示:
$ od -Ad test.txt 
0000000 061141 062143 030440 031462 032464 000012
0000011

实例3
不显示第一列偏移地址:
$ od -An test.txt 
 061141 062143 030440 031462 032464 000012

实例4
以十六进制数输出,默认以四字节为一组(一列)显示:
$ od -tx test.txt 
0000000 64636261 33323120 000a3534
0000013

实例5
以十六进制数输出,每列输出1个字节:
$ od -tx1 test.txt
0000000 61 62 63 64 20 31 32 33 34 35 0a
0000013

实例6
以具名字符显示ASCII字符:
$ od -ta test.txt
0000000   a   b   c   d  sp   1   2   3   4   5  nl
0000013
以可打印字符或反斜杠表示的转义字符显示ASCII字符:
$ od -tc test.txt
0000000   a   b   c   d       1   2   3   4   5  \n
0000013

实例7
以十六进制数显示的同时显示原字符:
$ od -tcx1 test.txt
0000000   a   b   c   d       1   2   3   4   5  \n
         61  62  63  64  20  31  32  33  34  35  0a
0000013

实例8
指定每行显示512字节:
$ od -w8 -tc test.txt
0000000   a   b   c   d       1   2   3
0000010   4   5  \n
0000013

实例9
实现od命令输出时去除列与列之间的空格符的方法:
1	使用-An不输出偏移地址;
2 使用-v输出时不省略重复的数据;
3 使用-tx1以单个字节为一组按照十六进制输出,-w1每列输出一个字节;
4 最后通过管道传递给 awk 的标准输入,通过awk不换行输出所有行,拼接为一行输出。
$ od -An -w1 -tx1 test.txt|awk '{for(i=1;i<=NF;++i){printf "%s",$i}}'
616263642031323334350a

What are Modules, Components, and Services? What Are Their Differences?

Components are also known as building blocks. But what exactly is a component? A component is a software unit that can be independently replaced and upgraded. It has the following characteristics:

  1. It performs a specific function or provides certain services.
  2. It cannot operate independently and must function as part of a system.
  3. It is a physical concept, not a logical one.
  4. It can be maintained, upgraded, or replaced independently without affecting the entire system.

A component is a physically independent entity that can be maintained, upgraded, or replaced on its own. The purpose of creating a component diagram is to perform component-based design for the system, thinking about the physical division of the system, identifying which existing components can be reused, and determining which parts can be turned into components for reuse in future projects.

Question 1: When designing software, we often mention the term “module.” Is a module the same as a component?

Not necessarily. Everyone has different standards for what constitutes a “module.” Sometimes modules are divided based on business logic, and other times they are divided from a technical perspective. Modules are simply a way to divide software into parts for ease of explanation. You can refer to the characteristics of a component listed above to determine whether a “module” qualifies as a component.

Question 2: Software often uses layered design. Is each layer a component?

In most cases, each layer in a layered design is just a logical division and is not physically represented as separate files. In such cases, the layers are not components. However, the actual design may vary, and you can refer to the characteristics of a component to make a judgment.

Question 3: How do we distinguish between a “service” and a “component”?

A “component” refers to a software unit that will be used by other applications beyond the author’s control, but these applications cannot modify the component. In other words, an application using a component cannot alter the component’s source code, but it can extend the component in a predefined way to change its behavior.

Services and components share some similarities: both are used by external applications. In my view, the biggest difference between them lies in the fact that components are libraries used locally, such as JAR files, assemblies, DLLs, or source code imports. Services, on the other hand, are components external to the process, accessed by applications through mechanisms such as synchronous or asynchronous inter-process communication or remote interface calls (e.g., web services, messaging systems, RPC, or sockets).

Services can also call other services since a service is, in itself, an application.

You could map each service to a runtime process, though this is only an approximation. A service could consist of multiple processes, such as the main service application process and a database process used exclusively by that service.

Services can be deployed independently. If an application system is composed of multiple libraries within a single process, any modification to one component requires redeployment of the entire application. However, if the system is divided into multiple services, only the modified service needs to be redeployed—unless changes were made to its exposed interface.

Another outcome of implementing components as services is the availability of more explicit component interfaces.

Compared to in-process calls, remote service calls are more expensive in terms of performance.

References:

  • “Microservices” by Martin Fowler
  • “Inversion of Control Containers and the Dependency Injection pattern” by Martin Fowler
  • “Fireball: UML and the War for Requirements Analysis”

The 21-Day Rule

The 21-day rule is a method for developing good habits through 21 days of repeated, correct practice.

Research shows that it takes 21 days for the brain to build a new neural pathway. As a result, human behavior, when repeated for more than 21 days, tends to form a habit. If repeated for over 90 days, it forms a stable habit.

Habit formation can be broken down into three stages:

  1. Stage 1 (Days 1-7): During this phase, you need to constantly remind yourself to change and intentionally push yourself to do so. If you don’t stay alert, old bad habits or negative emotions may resurface and pull you back. At this point, you might feel uncomfortable or unnatural, but these feelings are normal.
  2. Stage 2 (Days 7-21): After about a week of conscious effort, you’ll start to feel more comfortable with the new habit. However, you shouldn’t let your guard down yet. If you’re not careful, old habits could still disrupt your progress, so it’s important to continue reminding yourself to stay on track.
  3. Stage 3 (Days 21-90): This is the stabilization phase. By now, the new habit becomes a natural part of your life. You no longer need to make a conscious effort—it feels as effortless as checking the time on your watch.

Remember, only after a habit becomes second nature can your subconscious fully accept it and work in your favor. You may not yet realize the power of persistence, but it’s crucial to stay patient and persistent.

However, bad habits can be deeply ingrained due to repeated actions or suggestions over 90 or 100 times, sometimes more. Breaking these habits may require more effort, but the solution is simple: just stop doing them.

Many experiments and real-world practices have shown that through continuous repetition, habits and beliefs can be changed. While changing these habits or beliefs can feel uncomfortable and even undesirable, it’s important to persevere. Our actions are driven by beliefs and habits. Without changing the negative ones, our behaviors may also become harmful, leading to unwanted outcomes. Therefore, don’t give up just because change feels difficult—sometimes external pressure is necessary, especially in the beginning. As the saying goes, “The first step is always the hardest.” If you can’t start, the rest is impossible. Keep in mind, when changing any belief or habit, repetition is key—21 days or more. Believe that no habit or belief is beyond change, unless you accept failure.

To successfully change your beliefs or habits, you must follow the 21-day rule. Keep in mind three key points:

  1. Follow the three stages of habit formation.
  2. Be patient—new beliefs and habits require at least 21 days, and sometimes more.
  3. Repeat the practice consistently during this 21-day period.

Psychologist Ericsson’s research shows that the key factor determining excellence isn’t talent or experience, but the level of deliberate practice. Deliberate practice is designed specifically to improve performance by pushing you out of your comfort zone, forcing you to practice in ways that continually challenge and enhance your abilities. For example, football enthusiasts may simply enjoy the game, average players stick to routine training and matches, but top players are constantly aware of their weaknesses and challenge themselves with difficult, uncomfortable drills to improve.

Success comes from persistence. If you are nurturing your bad habits and are unwilling to change, ask yourself: do you want to fail or succeed? Do you want to waste away in boredom, or live a life full of energy and purpose?

If you want success, if you want a happy and fulfilling life, you have no other choice but to act now. How hard is it, really? Think of it as climbing a towering staircase—just focus on stepping up to the next step.

As you keep stepping forward, you’ll soon find yourself with a breathtaking view. Every practice is another step upward, and by focusing on the next step without stopping, you’ll eventually reach the top.

Linux xargs Command Passes Arguments to Other Commands

Description: xargs is used to pass arguments to other commands and is an essential component for building one-liner commands.

Syntax:
xargs [OPTIONS] [COMMAND]

Overview:
xargs takes input from stdin, separated by spaces or newline characters, and passes it as space-separated arguments to other commands. However, be careful when filenames or strings contain spaces, as xargs may misinterpret them.

Options:

  • -0, --null: Default option. If stdin contains special characters like backticks (), backslashes (\), or spaces, xargs` restores them to regular characters.
  • -a, --arg-file=FILE: Reads input from the specified file instead of stdin.
  • -d, --delimiter=DEL: Specifies the delimiter to separate input. By default, xargs uses spaces and newlines, outputting arguments separated by spaces.
  • -E EOF_STR: Sets an end-of-input string. If none is specified, input has no terminator. EOF_STR must be a separate field (i.e., space or newline separated).
  • -e, --eof[=EOF_STR]: Same as -E, but non-POSIX compliant. Use -E if available.
  • -I REPLACE_STR: Assigns each argument to the specified placeholder (e.g., {}, $, @). Useful for positioning arguments when there are multiple parameters. For example:find . -name "*.txt" | xargs -I {} cp {} /tmp/{}.bak
  • -i, --replace[=REPLACE_STR]: Same as -I, but REPLACE_STR is optional and defaults to {}. Use -I for POSIX compliance.
  • -L MAX_LINES: Limits the number of input lines per execution, implying the -x option.
  • -l, --max-lines[=MAX_LINES]: Same as -L. Defaults to 1 line. Use -L for POSIX compliance.
  • -n, --max-args=MAX_ARGS: Specifies the maximum number of arguments to pass to the command at once.
  • -o, --open-tty: Reopens stdin to /dev/TTY before running the command in a subprocess, useful for interactive applications.
  • -P, --max-procs=MAX_PROCS: Sets the maximum number of parallel processes. Default is 1. Use with -n or -L for batch processing.
  • -p, --interactive: Prompts the user for confirmation before executing each command.
  • --process-slot-var=NAME: Sets an environment variable with a unique value for each running subprocess. Once a process finishes, the value is reused.
  • -r, --no-run-if-empty: Stops xargs from running if there is no input. This is the default behavior.
  • -s, --max-chars=MAX_CHARS: Limits the maximum number of characters (including command, spaces, and newlines) in the command.
  • --show-limits: Displays the system’s command-line length limitations.
  • -t, --verbose: Prints the command to stderr before executing it.
  • -x, --exit: Exits if the command line exceeds the specified character limit (-s).
  • --help: Displays help information.
  • --version: Displays version information.

Parameters:

  • COMMAND: The command string to execute.

Examples:

Example 1
Some commands don’t accept piped arguments directly. Use xargs to pass them:

# Incorrect: `ls` cannot accept piped input directly
find /sbin -perm +700 | ls -l

# Correct: use `xargs` to pass arguments to `ls`
find /sbin -perm +700 | xargs ls -l

Example 2
Show system command-line length limitations:

$ xargs --show-limits

Example 3
Restore shell special characters like backticks:

$ echo '`0123`4 56789' | xargs -t echo

Example 4
Set the delimiter for reading input as a comma:

$ echo 01234 , 56789 | xargs -E ","

Example 5
Solve “argument list too long” errors when working with many files:

# Add a suffix to all files in the current directory
ls | xargs -t -i mv {} {}.bak

Example 6
Set how many lines to pass as arguments at a time:

$ echo -e "01234\n56789\n01234" | xargs -t -L 2 echo

Example 7
Merge multi-line input into a single line:

$ cat test.txt | xargs

Example 8
Kill processes in combination with ps, grep, awk, and kill:

$ ps -ef | grep spp | awk '{printf "%s ",$2}' | xargs kill -9

Linux命令xargs给其他命令传递参数

xargs给其他命令传递参数

功能说明:给其他命令传递参数,是构建单行命令的重要组件之一。

语  法:xargs [OPTIONS] [COMMAND]

补充说明:xargs可以将stdin中以空格或换行符进行分隔的数据,形成以空格分隔的参数(arguments),传递给其他命令。注意,因为以空格作为分隔符,所以有一些文件名或者其他意义的字符串内含有空格的时,xargs可能会误判。

    项:

-0, –null       默认选项。如果输入的stdin含有特殊字符,例如反引号 `、反斜杠 \、空格等字符时,xargs将它还原成一般字符。

-a, –arg-file=FILE    从指定的文件FILE中读取输入内容而不是从stdin

-d, –delimiter=DEL  指定xargs处理输入内容时的分隔符。xargs处理输入内容时默认使用空格和换行符作为分隔符,输出arguments时按空格分隔

-E EOF_STR      EOF_STR的意思是end of file string,表示输入结束的字符串。如果没有EOF_STR则表示输入没有结束符。注意,结束标识符必须要是单独的字段,即以空格或者换行符分隔开来的字段。

-e, –eof[=EOF_STR]       作用等同于-E选项。该选项不符合POSIX标准且EOF_STR是可选的。与-E选项不同时,以-E选项为准。

-I REPLACE_STR    将xargs输出的每一项参数单独赋值给后面的命令,参数需要用指定的替代字符串REPLACE_STR代替。REPLACE_STR可以使用{}、$、@ 等符号,其主要作用是当xargs命令后有多个参数时,用于调整参数位置。例如备份以 txt 为后缀的文件:find . -name “*.txt” | xargs -I {}  cp {} /tmp/{}.bak

-i, –replace[=REPLACE_STR]     作用同 -I 选项,REPLACE_STR是可选的,缺省为 {}。建议使用 -I 选项,因为其符合POSIX标准,而该选项不符合POSIX标准。

-L MAX_LINES        限定最大输入行数。隐含了 -x 选项。

-l, –max-lines[=MAX_LINES]     作用同 -L 选项,MAX_LINES 是可选的,缺省为1。建议使用 -L 选项,因为其符合 POSIX 标准,而该选项不符合POSIX标准。

-n, –max-args=MAX_ARGS         表示命令在执行的时候一次使用参数的最大个数。

-o, –open-tty       在执行命令之前,在子进程中重新打开stdin作为/dev/TTY。如果你希望xargs运行交互式命令行应用程序,这是非常有用的。

-P, –max-procs=MAX_PROCS     每次运行的最大进程数,默认值为1。如果MAX_PROCS为 0,xargs将一次运行尽可能多的进程。一般和-n或-L选项一起使用。

-p, –interactive   每次执行一个argument的时候询问一次用户。

–process-slot-var=NAME       将指定的环境变量设置为每个正在运行的子进程中的唯一值。一旦子进程退出,将重用该值。例如,这可以用于初始负荷分配方案。

-r, –no-run-if-empty         默认选项。当 xargs 的输入为空的时候则停止xargs,不用再去执行后面的命令了。

-s, –max-chars=MAX_CHARS     命令行的最大字符数,指的是xargs后面那个命令的最大字符个数,包括命令字符串本身、空格符和换行符。每个参数单独传入xargs后面的命令。

–show-limits      显示操作系统对命令行长度的限制。

-t, –verbose     先打印要执行的命令到标准错误输出,然后再执行。

-x, –exit              配合 -s 使用,当命令行字符数大于 -s 指定的数值时,退出 xargs。

–help           显示帮助信息并退出。

–version       显示版本信息并退出

    数:

COMMAND       命令字符串

    例:

实例1
很多命令不支持使用管道|来传递参数,此时可以使用xargs来传递参数,例如常用的ls命令:
 # 错误示例,因为标准输入不能作为ls的参数
find /sbin -perm +700 | ls -l
# 正确示例,使用xargs来传递参数
find /sbin -perm +700 | xargs ls -l

实例2
显示操作系统对命令行长度的限制信息:
$ xargs --show-limits
您的环境变量占有 2226 个字节
此系统的参数长度 POSIX 上限: 2092878
所有系统中所允许的最小参数长度 POSIX 上限: 4096
我们实际能用的最大命令长度: 2090652
我们实际能用的命令缓冲区的大小: 131072
最大并行数(--max-procs 不得大于该值):2147483647

设置命令行的最大字符数:
$ echo "01234 56789" | xargs -t -s 11
echo 01234
01234
echo 56789
56789

实例3
将 Shell 的反引号特殊字符还原为一般字符:
$ echo '`0123`4 56789' | xargs -t echo
echo '`0123`4' 56789
`0123`4 56789
其中-t选项的作用是,让xargs先打印要执行的命令到标准错误输出,然后再执行。因为反引号在 Shell 中会将 01234 作为一个命令来执行,但是 01234 不是一个命令。如果直接执行如下命令会报错:
$ echo `0123`4 56789
0123:未找到命令
4 56789

实例4
设置 xargs 读入参数时的结束标识为逗号",":
$ echo 01234 , 56789 | xargs -E ","
01234
注意,结束标识符必须要是单独的字段,即以空格或者换行符分隔开来的字段。

实例5
使用 rm、mv 等命令同时操作多个文件时,有时会报 “argument list too long” 参数列表过长的错误,此时可以使用 xargs 来解决这个错误。xargs 将标准输入的字符串分隔后,作为参数传递给后面的命令。例如:
# 给当前目录的所有文件添加后缀名
ls | xargs -t -i mv {} {}.bak
# 选择符合条件的文件
ls | grep -E "201701|201702|201703" | xargs -I {} mv {} {}.bak

实例6
设置标准输入中每次多少行作为xargs后面的命令的参数:
$ echo -e "01234\n56789\n01234" | xargs -t -L 2 echo
echo 01234 56789 
01234 56789
echo 01234 
01234
默认情况下-L选项的值是1,也就是将标准输入中所有行的归并到一行一次性传给xargs后面的命令执行:
$ echo -e "01234\n56789\n01234" | xargs -t echo
echo 01234 56789 01234
01234 56789 01234

实例7
将文件内容以空格分隔合并为一行输出:
# 列出文件内容
$ cat test.txt
a b c d e
f g h i j 
k l m n o
# 多行输入合并为一行输出
$ cat test.txt | xargs
a b c d e f g h i j k l m n o
# 相当于
$ cat test.txt | xargs echo
a b c d e f g h i j k l m n o

实例8
与ps、grep、awk和kill结合,强制终止指定进程:
$ ps -ef | grep spp | awk '{printf "%s ",$2}' | xargs kill -9
1
其中
ps -ef|grep spp用于查找进程名字或描述信息中包含 spp 子字符串的进程
awk '{printf "%s ",$2}将目标进程 ID 打印输出
xargs kill -9则将目标进程 ID 作为参数传递给kill -9用于杀死进程

Fix for Unable to Create Symlinks in Shared Folders Between VirtualBox Linux VM and Windows Host

Software Setup:

  • VirtualBox VM: Ubuntu 22
  • Host: Windows 10

When attempting to create symbolic links in a shared folder within an Ubuntu 22 VirtualBox VM, you may encounter several errors:

1.Error during npm install esbuild:

    Error: EPERM: operation not permitted, symlink '../esbuild/bin/esbuild' -> '/var/www/reverb.test/node_modules/.bin/esbuild'

    2.Error when creating symlinks in PHP:

    symlink(): Operation not permitted

    3.Error when running the ln command:

    ln: failed to create symbolic link 'xxx.so.0': Read-only file system

    4.Error when running the cp command:

    cannot create symbolic link `xxxx': Read-only file system

    5.Another error during cp:

    cannot create symbolic link `xxxx': Protocol error

    These issues occur because VirtualBox restricts the creation of symbolic links in shared folders for security reasons. To allow symbolic link creation, you’ll need to enable the option VBoxInternal2/SharedFoldersEnableSymlinksCreate/<share_folder_name>. For Vagrant users, this option is automatically configured when running vagrant up. For VirtualBox users, you can manually enable this setting using the VBoxManage.exe tool, which is located in the same directory as the VirtualBox GUI on Windows.

    Step-by-Step Guide:

    1.Close VirtualBox.

    2.Open a command prompt with administrator privileges and navigate to the VirtualBox installation directory (e.g., cd C:\Programs\Oracle\VirtualBox). Then, execute the following command:

    VBoxManage setextradata YOURVMNAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/YOURSHAREFOLDERNAME 1

    1) Replace YOURVMNAME with the name of your Ubuntu VM.

    2) Replace YOURSHAREFOLDERNAME with the name of the shared folder, which corresponds to the “Shared Folder Name” you set in VirtualBox, not the actual folder path or its name.

    Example:

    VBoxManage setextradata Ubuntu22 VBoxInternal2/SharedFoldersEnableSymlinksCreate/share 1

    3 Run VirtualBox as an administrator to apply the changes.

    References:

    VirtualBox Linux虚拟机与Windows宿主机的共享文件夹里面无法创建符号链接的解决方法

    我的软件环境:

    VirtualBox虚拟机:Ubuntu 22

    宿主机:Windows 10

    在Ubuntu 22虚拟机的共享文件夹中创建符号链接时,可能的报错有以下几种:

    第一种,在运行npm install esbuild时报错:

    Error: EPERM: operation not permitted, symlink '../esbuild/bin/esbuild' -> '/var/www/reverb.test/node_modules/.bin/esbuild'

    第二种,执行PHP代码创建符号链接时报错:

    symlink(): Operation not permitted

    第三种,运行ln命令时报错:

    ln: failed to create symbolic link 'xxx.so.0': Read-only file system

    第四种,运行cp命令时报错:

    cannot create symbolic link `xxxx':Read-only file system

    第五种,运行cp命令时报错:

    cannot create symbolic link `xxxx': Protocol error

    ……

    原因是VirtualBox从安全角度出发,限制了软链接(符号链接)的创建。如果想正常使用符号链接,需要对虚拟机设置 VBoxInternal2/SharedFoldersEnableSymlinksCreate/<share_folder_name> 这个选项。这里顺带一提,对于 vagrant,这个选项只要你运行 vagrant up之后这个选项便会自动设置好。可以使用 VboxManage.exe设置这个选项 ,这个工具在 Windows 下是和 VirtualBox 图形界面程序在一个相同目录的。以下是详细步骤:

    1 关闭 VirtualBox。

    2 以管理员身份打开命令行窗口,cd到VirtualBox安装目录(例如cd C:\Programs\Oracle\VirtualBox),执行如下命令:

    VBoxManage setextradata YOURVMNAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/YOURSHAREFOLDERNAME 1

    其中:

    • YOURVMNAME填写ubuntu 虚拟机系统的名称
    • YOURSHAREFOLDERNAME填写共享文件夹的名称,注意这个共享文件夹名称是你在 VirtualBox 中设置共享时对应的“共享文件夹名称”一栏的内容,不是共享文件夹的路径或者文件夹的名称。

    例如:

    VBoxManage setextradata Ubuntu22 VBoxInternal2/SharedFoldersEnableSymlinksCreate/share 1

    3 以管理员身份运行VirtualBox

    参考

    https://blog.csdn.net/tekenuo/article/details/82386552

    https://tty.moe/blog/virtualbox-symlink-windows