Linux xargs Command Passes Arguments to Other Commands

Description: xargs is used to pass arguments to other commands and is an essential component for building one-liner commands.

Syntax:
xargs [OPTIONS] [COMMAND]

Overview:
xargs takes input from stdin, separated by spaces or newline characters, and passes it as space-separated arguments to other commands. However, be careful when filenames or strings contain spaces, as xargs may misinterpret them.

Options:

  • -0, --null: Default option. If stdin contains special characters like backticks (), backslashes (\), or spaces, xargs` restores them to regular characters.
  • -a, --arg-file=FILE: Reads input from the specified file instead of stdin.
  • -d, --delimiter=DEL: Specifies the delimiter to separate input. By default, xargs uses spaces and newlines, outputting arguments separated by spaces.
  • -E EOF_STR: Sets an end-of-input string. If none is specified, input has no terminator. EOF_STR must be a separate field (i.e., space or newline separated).
  • -e, --eof[=EOF_STR]: Same as -E, but non-POSIX compliant. Use -E if available.
  • -I REPLACE_STR: Assigns each argument to the specified placeholder (e.g., {}, $, @). Useful for positioning arguments when there are multiple parameters. For example:find . -name "*.txt" | xargs -I {} cp {} /tmp/{}.bak
  • -i, --replace[=REPLACE_STR]: Same as -I, but REPLACE_STR is optional and defaults to {}. Use -I for POSIX compliance.
  • -L MAX_LINES: Limits the number of input lines per execution, implying the -x option.
  • -l, --max-lines[=MAX_LINES]: Same as -L. Defaults to 1 line. Use -L for POSIX compliance.
  • -n, --max-args=MAX_ARGS: Specifies the maximum number of arguments to pass to the command at once.
  • -o, --open-tty: Reopens stdin to /dev/TTY before running the command in a subprocess, useful for interactive applications.
  • -P, --max-procs=MAX_PROCS: Sets the maximum number of parallel processes. Default is 1. Use with -n or -L for batch processing.
  • -p, --interactive: Prompts the user for confirmation before executing each command.
  • --process-slot-var=NAME: Sets an environment variable with a unique value for each running subprocess. Once a process finishes, the value is reused.
  • -r, --no-run-if-empty: Stops xargs from running if there is no input. This is the default behavior.
  • -s, --max-chars=MAX_CHARS: Limits the maximum number of characters (including command, spaces, and newlines) in the command.
  • --show-limits: Displays the system’s command-line length limitations.
  • -t, --verbose: Prints the command to stderr before executing it.
  • -x, --exit: Exits if the command line exceeds the specified character limit (-s).
  • --help: Displays help information.
  • --version: Displays version information.

Parameters:

  • COMMAND: The command string to execute.

Examples:

Example 1
Some commands don’t accept piped arguments directly. Use xargs to pass them:

# Incorrect: `ls` cannot accept piped input directly
find /sbin -perm +700 | ls -l

# Correct: use `xargs` to pass arguments to `ls`
find /sbin -perm +700 | xargs ls -l

Example 2
Show system command-line length limitations:

$ xargs --show-limits

Example 3
Restore shell special characters like backticks:

$ echo '`0123`4 56789' | xargs -t echo

Example 4
Set the delimiter for reading input as a comma:

$ echo 01234 , 56789 | xargs -E ","

Example 5
Solve “argument list too long” errors when working with many files:

# Add a suffix to all files in the current directory
ls | xargs -t -i mv {} {}.bak

Example 6
Set how many lines to pass as arguments at a time:

$ echo -e "01234\n56789\n01234" | xargs -t -L 2 echo

Example 7
Merge multi-line input into a single line:

$ cat test.txt | xargs

Example 8
Kill processes in combination with ps, grep, awk, and kill:

$ ps -ef | grep spp | awk '{printf "%s ",$2}' | xargs kill -9

Linux命令xargs给其他命令传递参数

xargs给其他命令传递参数

功能说明:给其他命令传递参数,是构建单行命令的重要组件之一。

语  法:xargs [OPTIONS] [COMMAND]

补充说明:xargs可以将stdin中以空格或换行符进行分隔的数据,形成以空格分隔的参数(arguments),传递给其他命令。注意,因为以空格作为分隔符,所以有一些文件名或者其他意义的字符串内含有空格的时,xargs可能会误判。

    项:

-0, –null       默认选项。如果输入的stdin含有特殊字符,例如反引号 `、反斜杠 \、空格等字符时,xargs将它还原成一般字符。

-a, –arg-file=FILE    从指定的文件FILE中读取输入内容而不是从stdin

-d, –delimiter=DEL  指定xargs处理输入内容时的分隔符。xargs处理输入内容时默认使用空格和换行符作为分隔符,输出arguments时按空格分隔

-E EOF_STR      EOF_STR的意思是end of file string,表示输入结束的字符串。如果没有EOF_STR则表示输入没有结束符。注意,结束标识符必须要是单独的字段,即以空格或者换行符分隔开来的字段。

-e, –eof[=EOF_STR]       作用等同于-E选项。该选项不符合POSIX标准且EOF_STR是可选的。与-E选项不同时,以-E选项为准。

-I REPLACE_STR    将xargs输出的每一项参数单独赋值给后面的命令,参数需要用指定的替代字符串REPLACE_STR代替。REPLACE_STR可以使用{}、$、@ 等符号,其主要作用是当xargs命令后有多个参数时,用于调整参数位置。例如备份以 txt 为后缀的文件:find . -name “*.txt” | xargs -I {}  cp {} /tmp/{}.bak

-i, –replace[=REPLACE_STR]     作用同 -I 选项,REPLACE_STR是可选的,缺省为 {}。建议使用 -I 选项,因为其符合POSIX标准,而该选项不符合POSIX标准。

-L MAX_LINES        限定最大输入行数。隐含了 -x 选项。

-l, –max-lines[=MAX_LINES]     作用同 -L 选项,MAX_LINES 是可选的,缺省为1。建议使用 -L 选项,因为其符合 POSIX 标准,而该选项不符合POSIX标准。

-n, –max-args=MAX_ARGS         表示命令在执行的时候一次使用参数的最大个数。

-o, –open-tty       在执行命令之前,在子进程中重新打开stdin作为/dev/TTY。如果你希望xargs运行交互式命令行应用程序,这是非常有用的。

-P, –max-procs=MAX_PROCS     每次运行的最大进程数,默认值为1。如果MAX_PROCS为 0,xargs将一次运行尽可能多的进程。一般和-n或-L选项一起使用。

-p, –interactive   每次执行一个argument的时候询问一次用户。

–process-slot-var=NAME       将指定的环境变量设置为每个正在运行的子进程中的唯一值。一旦子进程退出,将重用该值。例如,这可以用于初始负荷分配方案。

-r, –no-run-if-empty         默认选项。当 xargs 的输入为空的时候则停止xargs,不用再去执行后面的命令了。

-s, –max-chars=MAX_CHARS     命令行的最大字符数,指的是xargs后面那个命令的最大字符个数,包括命令字符串本身、空格符和换行符。每个参数单独传入xargs后面的命令。

–show-limits      显示操作系统对命令行长度的限制。

-t, –verbose     先打印要执行的命令到标准错误输出,然后再执行。

-x, –exit              配合 -s 使用,当命令行字符数大于 -s 指定的数值时,退出 xargs。

–help           显示帮助信息并退出。

–version       显示版本信息并退出

    数:

COMMAND       命令字符串

    例:

实例1
很多命令不支持使用管道|来传递参数,此时可以使用xargs来传递参数,例如常用的ls命令:
 # 错误示例,因为标准输入不能作为ls的参数
find /sbin -perm +700 | ls -l
# 正确示例,使用xargs来传递参数
find /sbin -perm +700 | xargs ls -l

实例2
显示操作系统对命令行长度的限制信息:
$ xargs --show-limits
您的环境变量占有 2226 个字节
此系统的参数长度 POSIX 上限: 2092878
所有系统中所允许的最小参数长度 POSIX 上限: 4096
我们实际能用的最大命令长度: 2090652
我们实际能用的命令缓冲区的大小: 131072
最大并行数(--max-procs 不得大于该值):2147483647

设置命令行的最大字符数:
$ echo "01234 56789" | xargs -t -s 11
echo 01234
01234
echo 56789
56789

实例3
将 Shell 的反引号特殊字符还原为一般字符:
$ echo '`0123`4 56789' | xargs -t echo
echo '`0123`4' 56789
`0123`4 56789
其中-t选项的作用是,让xargs先打印要执行的命令到标准错误输出,然后再执行。因为反引号在 Shell 中会将 01234 作为一个命令来执行,但是 01234 不是一个命令。如果直接执行如下命令会报错:
$ echo `0123`4 56789
0123:未找到命令
4 56789

实例4
设置 xargs 读入参数时的结束标识为逗号",":
$ echo 01234 , 56789 | xargs -E ","
01234
注意,结束标识符必须要是单独的字段,即以空格或者换行符分隔开来的字段。

实例5
使用 rm、mv 等命令同时操作多个文件时,有时会报 “argument list too long” 参数列表过长的错误,此时可以使用 xargs 来解决这个错误。xargs 将标准输入的字符串分隔后,作为参数传递给后面的命令。例如:
# 给当前目录的所有文件添加后缀名
ls | xargs -t -i mv {} {}.bak
# 选择符合条件的文件
ls | grep -E "201701|201702|201703" | xargs -I {} mv {} {}.bak

实例6
设置标准输入中每次多少行作为xargs后面的命令的参数:
$ echo -e "01234\n56789\n01234" | xargs -t -L 2 echo
echo 01234 56789 
01234 56789
echo 01234 
01234
默认情况下-L选项的值是1,也就是将标准输入中所有行的归并到一行一次性传给xargs后面的命令执行:
$ echo -e "01234\n56789\n01234" | xargs -t echo
echo 01234 56789 01234
01234 56789 01234

实例7
将文件内容以空格分隔合并为一行输出:
# 列出文件内容
$ cat test.txt
a b c d e
f g h i j 
k l m n o
# 多行输入合并为一行输出
$ cat test.txt | xargs
a b c d e f g h i j k l m n o
# 相当于
$ cat test.txt | xargs echo
a b c d e f g h i j k l m n o

实例8
与ps、grep、awk和kill结合,强制终止指定进程:
$ ps -ef | grep spp | awk '{printf "%s ",$2}' | xargs kill -9
1
其中
ps -ef|grep spp用于查找进程名字或描述信息中包含 spp 子字符串的进程
awk '{printf "%s ",$2}将目标进程 ID 打印输出
xargs kill -9则将目标进程 ID 作为参数传递给kill -9用于杀死进程

Fix for Unable to Create Symlinks in Shared Folders Between VirtualBox Linux VM and Windows Host

Software Setup:

  • VirtualBox VM: Ubuntu 22
  • Host: Windows 10

When attempting to create symbolic links in a shared folder within an Ubuntu 22 VirtualBox VM, you may encounter several errors:

1.Error during npm install esbuild:

    Error: EPERM: operation not permitted, symlink '../esbuild/bin/esbuild' -> '/var/www/reverb.test/node_modules/.bin/esbuild'

    2.Error when creating symlinks in PHP:

    symlink(): Operation not permitted

    3.Error when running the ln command:

    ln: failed to create symbolic link 'xxx.so.0': Read-only file system

    4.Error when running the cp command:

    cannot create symbolic link `xxxx': Read-only file system

    5.Another error during cp:

    cannot create symbolic link `xxxx': Protocol error

    These issues occur because VirtualBox restricts the creation of symbolic links in shared folders for security reasons. To allow symbolic link creation, you’ll need to enable the option VBoxInternal2/SharedFoldersEnableSymlinksCreate/<share_folder_name>. For Vagrant users, this option is automatically configured when running vagrant up. For VirtualBox users, you can manually enable this setting using the VBoxManage.exe tool, which is located in the same directory as the VirtualBox GUI on Windows.

    Step-by-Step Guide:

    1.Close VirtualBox.

    2.Open a command prompt with administrator privileges and navigate to the VirtualBox installation directory (e.g., cd C:\Programs\Oracle\VirtualBox). Then, execute the following command:

    VBoxManage setextradata YOURVMNAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/YOURSHAREFOLDERNAME 1

    1) Replace YOURVMNAME with the name of your Ubuntu VM.

    2) Replace YOURSHAREFOLDERNAME with the name of the shared folder, which corresponds to the “Shared Folder Name” you set in VirtualBox, not the actual folder path or its name.

    Example:

    VBoxManage setextradata Ubuntu22 VBoxInternal2/SharedFoldersEnableSymlinksCreate/share 1

    3 Run VirtualBox as an administrator to apply the changes.

    References:

    VirtualBox Linux虚拟机与Windows宿主机的共享文件夹里面无法创建符号链接的解决方法

    我的软件环境:

    VirtualBox虚拟机:Ubuntu 22

    宿主机:Windows 10

    在Ubuntu 22虚拟机的共享文件夹中创建符号链接时,可能的报错有以下几种:

    第一种,在运行npm install esbuild时报错:

    Error: EPERM: operation not permitted, symlink '../esbuild/bin/esbuild' -> '/var/www/reverb.test/node_modules/.bin/esbuild'

    第二种,执行PHP代码创建符号链接时报错:

    symlink(): Operation not permitted

    第三种,运行ln命令时报错:

    ln: failed to create symbolic link 'xxx.so.0': Read-only file system

    第四种,运行cp命令时报错:

    cannot create symbolic link `xxxx':Read-only file system

    第五种,运行cp命令时报错:

    cannot create symbolic link `xxxx': Protocol error

    ……

    原因是VirtualBox从安全角度出发,限制了软链接(符号链接)的创建。如果想正常使用符号链接,需要对虚拟机设置 VBoxInternal2/SharedFoldersEnableSymlinksCreate/<share_folder_name> 这个选项。这里顺带一提,对于 vagrant,这个选项只要你运行 vagrant up之后这个选项便会自动设置好。可以使用 VboxManage.exe设置这个选项 ,这个工具在 Windows 下是和 VirtualBox 图形界面程序在一个相同目录的。以下是详细步骤:

    1 关闭 VirtualBox。

    2 以管理员身份打开命令行窗口,cd到VirtualBox安装目录(例如cd C:\Programs\Oracle\VirtualBox),执行如下命令:

    VBoxManage setextradata YOURVMNAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/YOURSHAREFOLDERNAME 1

    其中:

    • YOURVMNAME填写ubuntu 虚拟机系统的名称
    • YOURSHAREFOLDERNAME填写共享文件夹的名称,注意这个共享文件夹名称是你在 VirtualBox 中设置共享时对应的“共享文件夹名称”一栏的内容,不是共享文件夹的路径或者文件夹的名称。

    例如:

    VBoxManage setextradata Ubuntu22 VBoxInternal2/SharedFoldersEnableSymlinksCreate/share 1

    3 以管理员身份运行VirtualBox

    参考

    https://blog.csdn.net/tekenuo/article/details/82386552

    https://tty.moe/blog/virtualbox-symlink-windows

    Why Multiple Elasticsearch Nodes Fail to Form a Cluster: master_not_discovered_exception or NotMasterException and How to Fix It

    If you’ve encountered a situation where multiple Elasticsearch nodes fail to automatically form a cluster and the logs show the error: “master_not_discovered_exception”, you are not alone.

    The Problem

    The root cause is often related to system administrators cloning virtual machines to create multiple Elasticsearch servers. When this happens, every Elasticsearch node ends up with the same node ID, and as a result, the cluster cannot successfully elect a master node.

    Verifying the Issue

    You can verify this issue by listing all the node IDs with the following command:

    GET /_cat/nodes?v&h=id,ip,name&full_id=true

    However, since the Elasticsearch cluster hasn’t formed, you need to query each node individually, like this:

    curl 192.168.110.111:9200/_cat/nodes?v&h=id,ip,name&full_id=true
    curl 192.168.110.112:9200/_cat/nodes?v&h=id,ip,name&full_id=true

    The Solution

    Elasticsearch requires each node to have a unique node ID. To fix this issue, you need to delete the index data on each node. If Elasticsearch was installed using the RPM package, the index data is usually stored in /var/lib/elasticsearch by default. After deleting the data, restart Elasticsearch, and it will generate a new, unique node ID for each node.

    Reference

    For further details, check the full article here: https://www.656463.com/wenda/jdbhjrjqNotMasterExceptionqgddxc_359.

    Fix for “An SSH Installation Couldn’t Be Found” Error When Connecting to a Remote Server in VS Code

    Operating System: Windows 10

    When using the Remote-SSH extension in VS Code for remote development, you might encounter the error message “An SSH installation couldn’t be found” during the first connection attempt to a remote server. This error indicates that the VS Code Remote-SSH extension cannot find ssh.exe in your Windows system.

    By default, VS Code looks for the ssh command in the system’s PATH environment variable. If it’s not found, this error occurs.

    One solution is to install the Git package on your Windows system, which includes ssh.exe. Then, you can specify the exact path to ssh.exe in the settings of the VS Code Remote-SSH extension by configuring the remote.SSH.path option. Here’s how:

    1. Install the Git package, which includes ssh.exe.
    2. In VS Code, press F1 -> type Remote-SSH: Settings and hit Enter. This will open the Remote-SSH configuration file.
    3. Locate the remote.SSH.path option and set its value to the absolute path of ssh.exe on your Windows system. For example:
    C:\Programs\Git\usr\bin\ssh.exe

    Alternatively, you can install the OpenSSH package, which also includes ssh.exe, and use that instead of the Git-provided version.

    Setting Up SSH Key Authentication and Communication with GitHub

    When connecting to an existing GitHub repository, you can use SSH keys (public and private) for authentication. For instructions on how to create an SSH key and add it to GitHub, refer to the official GitHub documentation: https://docs.github.com/cn/authentication/connecting-to-github-with-ssh.

    Key steps to note:

    1. You need to create a .ssh folder in your home directory and copy the SSH private key into it. You do not need to copy the public key into this folder. Instead, paste the contents of your public key into the Settings section of your GitHub account by following the steps here: Adding a new SSH key to your GitHub account.
    2. The file name of your SSH private key should be id_rsa, as this is one of the default names git looks for when locating the private key. If the file is named differently, you will encounter an error such as “[email protected]: Permission denied (publickey).” If this happens, rename your private key file to id_rsa.
    3. If you’re using a Linux-based OS like Ubuntu, ensure that no other users can read or write to the private key file. Run the following commands:
    chmod o-rwx ./id_rsa
    chmod g-rwx ./id_rsa

    After completing the setup, you can use the SSH private key in your .ssh folder for authentication and communication with GitHub. To verify the setup, open Git Bash and run the following command:

    ssh -T [email protected]

    If you see a message like this:

    Hi yourusername! You've successfully authenticated, but GitHub does not provide shell access.

    It means you’ve successfully set up SSH key authentication with GitHub.

    References:

    Windows 10 or Windows 11 Only Shows One Hard Drive After Reinstallation? Here’s the Fix

    If you have two hard drives in your system but only see one in “This PC” after reinstalling Windows 10 or 11, follow these steps to resolve the issue:

    1. Right-click on This PC and select Manage.
    2. Go to Device Manager under System Tools and check Disk drives. If both drives are listed, it means the second drive is recognized and there is no issue with the hardware.
    3. Now, open Storage > Disk Management.
    4. Right-click on the second hard drive and select Change Drive Letter and Paths. Assign a drive letter to the second drive.

    After doing this, the second drive should appear in This PC in File Explorer.

    Windows 10或Windows 11有两个硬盘,但重装系统后在“此电脑”里只显示一个的问题及解决方法

    右键点“此电脑”->系统工具->设备管理->磁盘驱动器,看看里面是否有两个硬盘驱动,如果是,说明第二块磁盘驱动没有问题,再打开存储->磁盘管理,右键点击第二块磁盘->更改驱动器号和路径,设置一下驱动器号,就可以在资源管理器的“此电脑”路径下出现了。

    How to Restore Files Deleted by Windows Defender on Windows 10 and Windows 11?

    For Windows 10:
    1.Open Windows Security Center
    2.Go to Virus & threat protection
    3.Navigate to Threat history
    4.Check the Detected threats
    5.Select Restore

    For Windows 11:
    1.Open Windows Security Center
    2.Go to Virus & threat protection
    3.Navigate to Protection history
    4.Check the Quarantined threats
    5.Select Restore

    Important Note
    Restoring quarantined files requires administrative privileges on your Windows system.