shell sed对文件的操作

  • 简介

  • sed命令

  • sed小技巧





一、简介

在shell脚本编写中,时常会用到对文件的相关操作,比如增加内容,修改内容,删除部分内容,查看部分内容等,但是上述举例的这些操作一般都是需要在文本编辑器中才能操作,常用的文本编辑器如:gedit、vim、nano等又是交互式文本编辑器,脚本无法自己独立完成,必须有人参与才可以完成。如果这样的话又违背了我们编写脚本的意愿(全部由机器来完成,减少人的工作压力,提升工作效率)。emm….如何才能让这些操作全部脚本自己就搞定,而不需要人的参与,而且又能按照我们的脚本预案来完成呢?

为了解决上述问题,linux为大家提供了一些命令,比如Perl、sed等命令,今天我就着重为大家介绍一下sed命令。

二、sed命令

sed是linux中提供的一个外部命令,它是一个行(流)编辑器,非交互式的对文件内容进行增删改查的操作,使用者只能在命令行输入编辑命令、指定文件名,然后在屏幕上查看输出。它和文本编辑器有本质的区别。


  1. 区别是:
  2.  
  3. 文本编辑器: 编辑对象是文件
  4.  
  5. 行编辑器:编辑对象是文件中的行

也就是前者一次处理一个文本,而后者是一次处理一个文本中的一行。这个是我们应该弄清楚且必须牢记的,否者可能无法理解sed的运行原理和使用精髓。

2.1)sed数据处理原理

2.2)sed命令

sed 命令

语法:

sed [options] ‘{command}[flags]’ [filename]


  1. 命令选项
  2. -e script 将脚本中指定的命令添加到处理输入时执行的命令中 多条件,一行中要有多个操作
  3. -f script 将文件中指定的命令添加到处理输入时执行的命令中
  4. -n 抑制自动输出
  5. -i 编辑文件内容
  6. -i.bak 修改时同时创建.bak备份文件。
  7. -r 使用扩展的正则表达式
  8. ! 取反 (跟在模式条件后与shell有所区别)
  9.  
  10.  
  11. sed常用内部命令
  12. a 在匹配后面添加
  13. i 在匹配前面添加
  14. p 打印
  15. d 删除
  16. s 查找替换
  17. c 更改
  18. y 转换 N D P
  19.  
  20.  
  21. flags
  22. 数字 表示新文本替换的模式
  23. g: 表示用新文本替换现有文本的全部实例
  24. p: 表示打印原始的内容
  25. w filename: 将替换的结果写入文件

2.2.1)sed内部命令说明

演示实例文档


  1. [root@www ~]# cat data1
  2. 1 the quick brown fox jumps over the lazy dog.
  3. 2 the quick brown fox jumps over the lazy dog.
  4. 3 the quick brown fox jumps over the lazy dog.
  5. 4 the quick brown fox jumps over the lazy dog.
  6. 5 the quick brown fox jumps over the lazy dog.

文件内容增加操作,将数据追加到某个位置之后,使用命令a。

演示案例


  1. 在data1的每行后追加一行新数据内容: append data "haha"
  2. [root@www ~]# sed 'a\append data "haha"' data1
  3. 1 the quick brown fox jumps over the lazy dog.
  4. append data "haha"
  5. 2 the quick brown fox jumps over the lazy dog.
  6. append data "haha"
  7. 3 the quick brown fox jumps over the lazy dog.
  8. append data "haha"
  9. 4 the quick brown fox jumps over the lazy dog.
  10. append data "haha"
  11. 5 the quick brown fox jumps over the lazy dog.
  12. append data "haha"
  13.  
  14.  
  15. 在第二行后新开一行追加数据: append data "haha"
  16. [root@www ~]# sed '2a\append data "haha"' data1
  17. 1 the quick brown fox jumps over the lazy dog.
  18. 2 the quick brown fox jumps over the lazy dog.
  19. append data "haha"
  20. 3 the quick brown fox jumps over the lazy dog.
  21. 4 the quick brown fox jumps over the lazy dog.
  22. 5 the quick brown fox jumps over the lazy dog.
  23.  
  24. 在第二到四行每行后新开一行追加数据: append data "haha"
  25. [root@www ~]# sed '2,4a\append data "haha"' data1
  26. 1 the quick brown fox jumps over the lazy dog.
  27. 2 the quick brown fox jumps over the lazy dog.
  28. append data "haha"
  29. 3 the quick brown fox jumps over the lazy dog.
  30. append data "haha"
  31. 4 the quick brown fox jumps over the lazy dog.
  32. append data "haha"
  33. 5 the quick brown fox jumps over the lazy dog.
  34.  
  35. 匹配字符串追加: 找到包含"3 the"的行,在其后新开一行追加内容: append data "haha"
  36. [root@www ~]# sed '/3 the/a\append data "haha"' data1
  37. 1 the quick brown fox jumps over the lazy dog.
  38. 2 the quick brown fox jumps over the lazy dog.
  39. 3 the quick brown fox jumps over the lazy dog.
  40. append data "haha"
  41. 4 the quick brown fox jumps over the lazy dog.
  42. 5 the quick brown fox jumps over the lazy dog.
  43.  
  44. //开启匹配模式 /要匹配的字符串/

文件内容增加操作,将数据插入到某个位置之前,使用命令i。

演示案例


  1. 在data1的每行前插入一行新数据内容: insert data "haha"
  2. [root@www ~]# sed 'i\insert data "haha"' data1
  3. insert data "haha"
  4. 1 the quick brown fox jumps over the lazy dog.
  5. insert data "haha"
  6. 2 the quick brown fox jumps over the lazy dog.
  7. insert data "haha"
  8. 3 the quick brown fox jumps over the lazy dog.
  9. insert data "haha"
  10. 4 the quick brown fox jumps over the lazy dog.
  11. insert data "haha"
  12. 5 the quick brown fox jumps over the lazy dog.
  13.  
  14.  
  15. 在第二行前新开一行插入数据: insert data "haha"
  16. [root@www ~]# sed '2i\insert data "haha"' data1
  17. 1 the quick brown fox jumps over the lazy dog.
  18. insert data "haha"
  19. 2 the quick brown fox jumps over the lazy dog.
  20. 3 the quick brown fox jumps over the lazy dog.
  21. 4 the quick brown fox jumps over the lazy dog.
  22. 5 the quick brown fox jumps over the lazy dog.
  23.  
  24.  
  25. 在第二到四行每行前新开一行插入数据: insert data "haha"
  26. [root@www ~]# sed '2,4i\insert data "haha"' data1
  27. 1 the quick brown fox jumps over the lazy dog.
  28. insert data "haha"
  29. 2 the quick brown fox jumps over the lazy dog.
  30. insert data "haha"
  31. 3 the quick brown fox jumps over the lazy dog.
  32. insert data "haha"
  33. 4 the quick brown fox jumps over the lazy dog.
  34. 5 the quick brown fox jumps over the lazy dog.
  35.  
  36.  
  37. 匹配字符串插入: 找到包含"3 the"的行,在其前新开一行插入内容: insert data "haha"
  38. [root@www ~]# sed '/3 the/i\insert data "haha"' data1
  39. 1 the quick brown fox jumps over the lazy dog.
  40. 2 the quick brown fox jumps over the lazy dog.
  41. insert data "haha"
  42. 3 the quick brown fox jumps over the lazy dog.
  43. 4 the quick brown fox jumps over the lazy dog.
  44. 5 the quick brown fox jumps over the lazy dog.

文件内容修改操作—替换,将一行中匹配的内容替换为新的数据,使用命令s。

演示案例


  1. 从标准输出流中做替换,将test替换为text
  2. [root@www ~]# echo "this is a test" |sed 's/test/text/'
  3. this is a text
  4.  
  5. 将data1中每行的dog替换为cat
  6. [root@www ~]# sed 's/dog/cat/' data1
  7. 1 the quick brown fox jumps over the lazy cat.
  8. 2 the quick brown fox jumps over the lazy cat.
  9. 3 the quick brown fox jumps over the lazy cat.
  10. 4 the quick brown fox jumps over the lazy cat.
  11. 5 the quick brown fox jumps over the lazy cat.
  12.  
  13. 将data1中第二行的dog替换为cat
  14. [root@www ~]# sed '2s/dog/cat/' data1
  15. 1 the quick brown fox jumps over the lazy dog.
  16. 2 the quick brown fox jumps over the lazy cat.
  17. 3 the quick brown fox jumps over the lazy dog.
  18. 4 the quick brown fox jumps over the lazy dog.
  19. 5 the quick brown fox jumps over the lazy dog.
  20.  
  21. 将data1中第二到第四行的dog替换为cat
  22. [root@www ~]# sed '2,4s/dog/cat/' data1
  23. 1 the quick brown fox jumps over the lazy dog.
  24. 2 the quick brown fox jumps over the lazy cat.
  25. 3 the quick brown fox jumps over the lazy cat.
  26. 4 the quick brown fox jumps over the lazy cat.
  27. 5 the quick brown fox jumps over the lazy dog.
  28.  
  29. 匹配字符串替换:将包含字符串"3 the"的行中的dog替换为cat
  30. [root@www ~]# sed '/3 the/s/dog/cat/' data1
  31. 1 the quick brown fox jumps over the lazy dog.
  32. 2 the quick brown fox jumps over the lazy dog.
  33. 3 the quick brown fox jumps over the lazy cat.
  34. 4 the quick brown fox jumps over the lazy dog.
  35. 5 the quick brown fox jumps over the lazy dog.

文件内容修改操作—更改,将一行中匹配的内容替换为新的数据,使用命令c。

演示案例


  1. 将data1文件中的所有行的内容更改为: change data "data"
  2. [root@www ~]# sed 'c\change data "haha"' data1
  3. change data "haha"
  4. change data "haha"
  5. change data "haha"
  6. change data "haha"
  7. change data "haha"
  8.  
  9. 将data1文件第二行的内容更改为: change data "haha"
  10. [root@www ~]# sed '2c\change data "haha"' data1
  11. 1 the quick brown fox jumps over the lazy dog.
  12. change data "haha"
  13. 3 the quick brown fox jumps over the lazy dog.
  14. 4 the quick brown fox jumps over the lazy dog.
  15. 5 the quick brown fox jumps over the lazy dog.
  16.  
  17. 将data1文件中的第二、三、四行的内容更改为:change data "haha"
  18. [root@www ~]# sed '2,4c\change data "haha"' data1
  19. 1 the quick brown fox jumps over the lazy dog.
  20. change data "haha"
  21. 5 the quick brown fox jumps over the lazy dog.
  22.  
  23. 将data1文件中包含"3 the"的行内容更改为: change data "haha"
  24. [root@www ~]# sed '/3 the/c\change data "data"' data1
  25. 1 the quick brown fox jumps over the lazy dog.
  26. 2 the quick brown fox jumps over the lazy dog.
  27. change data "data"
  28. 4 the quick brown fox jumps over the lazy dog.
  29. 5 the quick brown fox jumps over the lazy dog.

文件内容修改操作—字符转换,将一行中匹配的内容替换为新的数据,使用命令y。

演示案例


  1. 将data1中的a b c字符转换为对应的 A B C字符
  2. [root@www ~]# sed 'y/abc/ABC/' data1
  3. 1 the quiCk Brown fox jumps over the lAzy dog.
  4. 2 the quiCk Brown fox jumps over the lAzy dog.
  5. 3 the quiCk Brown fox jumps over the lAzy dog.
  6. 4 the quiCk Brown fox jumps over the lAzy dog.
  7. 5 the quiCk Brown fox jumps over the lAzy dog.

文件内容删除,将文件中的指定数据删除,使用命令d。

演示案例


  1. 删除文件data1中的所有数据
  2. [root@www ~]# sed 'd' data1
  3.  
  4. 删除文件data1中的第三行数据
  5. [root@www ~]# sed '3d' data1
  6. 1 the quick brown fox jumps over the lazy dog.
  7. 2 the quick brown fox jumps over the lazy dog.
  8. 4 the quick brown fox jumps over the lazy dog.
  9. 5 the quick brown fox jumps over the lazy dog.
  10.  
  11. 删除文件data1第三到第四行的数据
  12. [root@www ~]# sed '3,4d' data1
  13. 1 the quick brown fox jumps over the lazy dog.
  14. 2 the quick brown fox jumps over the lazy dog.
  15. 5 the quick brown fox jumps over the lazy dog.
  16.  
  17. 删除文件data1中包含字符串"3 the"的行
  18. [root@www ~]# sed '/3 the/d' data1
  19. 1 the quick brown fox jumps over the lazy dog.
  20. 2 the quick brown fox jumps over the lazy dog.
  21. 4 the quick brown fox jumps over the lazy dog.
  22. 5 the quick brown fox jumps over the lazy dog.

文件内容查看,将文件内容输出到屏幕,使用命令p。

演示案例


  1. 打印data1文件内容
  2. [root@www ~]# sed 'p' data1
  3. 1 the quick brown fox jumps over the lazy dog.
  4. 1 the quick brown fox jumps over the lazy dog.
  5. 2 the quick brown fox jumps over the lazy dog.
  6. 2 the quick brown fox jumps over the lazy dog.
  7. 3 the quick brown fox jumps over the lazy dog.
  8. 3 the quick brown fox jumps over the lazy dog.
  9. 4 the quick brown fox jumps over the lazy dog.
  10. 4 the quick brown fox jumps over the lazy dog.
  11. 5 the quick brown fox jumps over the lazy dog.
  12. 5 the quick brown fox jumps over the lazy dog.
  13.  
  14. 打印data1文件第三行的内容
  15. [root@www ~]# sed '3p' data1
  16. 1 the quick brown fox jumps over the lazy dog.
  17. 2 the quick brown fox jumps over the lazy dog.
  18. 3 the quick brown fox jumps over the lazy dog.
  19. 3 the quick brown fox jumps over the lazy dog.
  20. 4 the quick brown fox jumps over the lazy dog.
  21. 5 the quick brown fox jumps over the lazy dog.
  22.  
  23. 打印data1文件第二、三、四行内容
  24. [root@www ~]# sed '2,4p' data1
  25. 1 the quick brown fox jumps over the lazy dog.
  26. 2 the quick brown fox jumps over the lazy dog.
  27. 2 the quick brown fox jumps over the lazy dog.
  28. 3 the quick brown fox jumps over the lazy dog.
  29. 3 the quick brown fox jumps over the lazy dog.
  30. 4 the quick brown fox jumps over the lazy dog.
  31. 4 the quick brown fox jumps over the lazy dog.
  32. 5 the quick brown fox jumps over the lazy dog.
  33.  
  34. 打印data1文件包含字符串"3 the"的行
  35. [root@www ~]# sed '/3 the/p' data1
  36. 1 the quick brown fox jumps over the lazy dog.
  37. 2 the quick brown fox jumps over the lazy dog.
  38. 3 the quick brown fox jumps over the lazy dog.
  39. 3 the quick brown fox jumps over the lazy dog.
  40. 4 the quick brown fox jumps over the lazy dog.
  41. 5 the quick brown fox jumps over the lazy dog.
  42.  
  43.  
  44. 可以看得出,打印内容是重复的行,原因是打印了指定文件内容一次,又将读入缓存的所有数据打印了一次,所以会看到这样的效果,
  45. 如果不想看到这样的结果,可以加命令选项-n抑制内存输出即可。

2.2.2)命令选项说明

在命令行中使用多个命令 -e


  1. 将brown替换为green dog替换为cat
  2. [root@www ~]# sed -e 's/brown/green/;s/dog/cat/' data1
  3. 1 the quick green fox jumps over the lazy cat.
  4. 2 the quick green fox jumps over the lazy cat.
  5. 3 the quick green fox jumps over the lazy cat.
  6. 4 the quick green fox jumps over the lazy cat.
  7. 5 the quick green fox jumps over the lazy cat.

从文件读取编辑器命令 -f 适用于日常重复执行的场景


  1. 1)将命令写入文件
  2. [root@www ~]# vim abc
  3. s/brown/green/
  4. s/dog/cat/
  5. s/fox/elephant/
  6.  
  7. 2)使用-f命令选项调用命令文件
  8. [root@www ~]# sed -f abc data1
  9. 1 the quick green elephant jumps over the lazy cat.
  10. 2 the quick green elephant jumps over the lazy cat.
  11. 3 the quick green elephant jumps over the lazy cat.
  12. 4 the quick green elephant jumps over the lazy cat.
  13. 5 the quick green elephant jumps over the lazy cat.

抑制内存输出 -n


  1. 打印data1文件的第二行到最后一行内容 $最后的意思
  2. [root@www ~]# sed -n '2,$p' data1
  3. 2 the quick brown fox jumps over the lazy dog.
  4. 3 the quick brown fox jumps over the lazy dog.
  5. 4 the quick brown fox jumps over the lazy dog.
  6. 5 the quick brown fox jumps over the lazy dog.

使用正则表达式 -r


  1. 打印data1中以字符串"3 the"开头的行内容
  2. [root@www ~]# sed -n -r '/^(3 the)/p' data1
  3. 3 the quick brown fox jumps over the lazy dog.

从上述的演示中,大家可以看出,数据处理只是在缓存中完成的,并没有实际修改文件内容,如果需要修改文件内容可以直接使用-i命令选项。在这里我需要说明的是-i是一个不可逆的操作,一旦修改,如果想复原就很困难,几乎不可能,所以建议大家在操作的时候可以备份一下源文件。-i命令选项提供了备份功能,比如参数使用-i.bak,那么在修改源文件的同时会先备份一个以.bak结尾的源文件,然后再进行修改操作。


  1. 1)查看文件列表,没有发现data1.bak
  2. [root@www ~]# ls
  3. abc apache data1 Dobby file node-v10.14.1 Python-3.7.1 soft1 vimset
  4.  
  5. 2)执行替换命令并修改文件
  6. [root@www ~]# sed -i.bak 's/brown/green/' data1
  7.  
  8. 3)发现文件夹中多了一个data1.bak文件
  9. [root@www ~]# ls
  10. abc data1 Dobby node-v10.14.1 soft1
  11. apache data1.bak file Python-3.7.1 vimset
  12.  
  13. 4)打印比较一下,发现data1已经被修改,data1.bak是源文件的备份。
  14. [root@www ~]# cat data1
  15. 1 the quick green fox jumps over the lazy dog.
  16. 2 the quick green fox jumps over the lazy dog.
  17. 3 the quick green fox jumps over the lazy dog.
  18. 4 the quick green fox jumps over the lazy dog.
  19. 5 the quick green fox jumps over the lazy dog.
  20. [root@www ~]# cat data1.bak
  21. 1 the quick brown fox jumps over the lazy dog.
  22. 2 the quick brown fox jumps over the lazy dog.
  23. 3 the quick brown fox jumps over the lazy dog.
  24. 4 the quick brown fox jumps over the lazy dog.
  25. 5 the quick brown fox jumps over the lazy dog.

2.2.3)标志


  1. 演示文档
  2. [root@www ~]# cat data2
  3. 1 the quick brown fox jumps over the lazy dog . dog
  4. 2 the quick brown fox jumps over the lazy dog . dog
  5. 3 the quick brown fox jumps over the lazy dog . dog
  6. 4 the quick brown fox jumps over the lazy dog . dog
  7. 5 the quick brown fox jumps over the lazy dog . dog

数字标志:此标志是一个非零正数,默认情况下,执行替换的时候,如果一行中有多个符合的字符串,如果没有标志位定义,那么只会替换第一个字符串,其他的就被忽略掉了,为了能精确替换,可以使用数字位做定义。


  1. 替换一行中的第二处dog为cat
  2. [root@www ~]# sed 's/dog/cat/2' data2
  3. 1 the quick brown fox jumps over the lazy dog . cat
  4. 2 the quick brown fox jumps over the lazy dog . cat
  5. 3 the quick brown fox jumps over the lazy dog . cat
  6. 4 the quick brown fox jumps over the lazy dog . cat
  7. 5 the quick brown fox jumps over the lazy dog . cat

g标志:将一行中的所有符合的字符串全部执行替换


  1. 将data1文件中的所有dog替换为cat
  2. [root@www ~]# sed 's/dog/cat/g' data2
  3. 1 the quick brown fox jumps over the lazy cat . cat
  4. 2 the quick brown fox jumps over the lazy cat . cat
  5. 3 the quick brown fox jumps over the lazy cat . cat
  6. 4 the quick brown fox jumps over the lazy cat . cat
  7. 5 the quick brown fox jumps over the lazy cat . cat

p标志:打印文本内容,类似于-p命令选项


  1. [root@www ~]# sed '3s/dog/cat/p' data2
  2. 1 the quick brown fox jumps over the lazy dog . dog
  3. 2 the quick brown fox jumps over the lazy dog . dog
  4. 3 the quick brown fox jumps over the lazy cat . dog
  5. 3 the quick brown fox jumps over the lazy cat . dog
  6. 4 the quick brown fox jumps over the lazy dog . dog
  7. 5 the quick brown fox jumps over the lazy dog . dog

w filename标志:将修改的内容存入filename文件中


  1. [root@www ~]# sed '3s/dog/cat/w text' data2
  2. 1 the quick brown fox jumps over the lazy dog . dog
  3. 2 the quick brown fox jumps over the lazy dog . dog
  4. 3 the quick brown fox jumps over the lazy cat . dog
  5. 4 the quick brown fox jumps over the lazy dog . dog
  6. 5 the quick brown fox jumps over the lazy dog . dog
  7.  
  8. 可以看出,将修改的第三行内容存在了text文件中
  9. [root@www ~]# cat text
  10. 3 the quick brown fox jumps over the lazy cat . dog

三、sed小技巧

$= 统计文本有多少行


  1. 统计data2有多少行
  2. [root@www ~]# sed -n '$=' data2
  3. 5
  4.  
  5. 打印data2内容时加上行号
  6. [root@www ~]# sed '=' data2
  7. 1
  8. 1 the quick brown fox jumps over the lazy dog . dog
  9. 2
  10. 2 the quick brown fox jumps over the lazy dog . dog
  11. 3
  12. 3 the quick brown fox jumps over the lazy dog . dog
  13. 4
  14. 4 the quick brown fox jumps over the lazy dog . dog
  15. 5
  16. 5 the quick brown fox jumps over the lazy dog . dog

 


  • 作者:合十
  • 发表时间:2021年9月4日 03:44
  • 更新时间:2024年3月29日 20:28
  • 所属分类:Shell脚本编程

Comments

该文章还未收到评论,点击下方评论框开始评论吧~