Questions tagged [sed]
"sed" ("stream editor") is a Unix utility that parses and transforms text files.
1,044
questions
323
votes
6
answers
518k
views
How to match whitespace in sed?
How can I match whitespace in sed? In my data I want to match all of 3+ subsequent whitespace characters (tab space) and replace them by 2 spaces. How can this be done?
- 9,576
193
votes
18
answers
161k
views
Removing ANSI color codes from text stream
Examining the output from
perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";'
in a text editor (e.g., vi) shows the following:
^[[37mABC
^[[0m
How would one ...
- 3,704
169
votes
4
answers
194k
views
How to match digits followed by a dot using sed?
I'm trying to use sed to substitute all the patterns with digits followed immediately by a dot (such as 3., 355.) by an empty string. So I try:
sed 's/\d+\.//g' file.txt
But it doesn't work. Why is ...
- 1,879
132
votes
18
answers
55k
views
Substitution in text file **without** regular expressions
I need to substitute some text inside a text file with a replacement. Usually I would do something like
sed -i 's/text/replacement/g' path/to/the/file
The problem is that both text and replacement ...
- 1,525
92
votes
6
answers
110k
views
use of alternation "|" in sed's regex
I am using sed, GNU sed version 4.2.1.
I want to use the alternation "|" symbol in a subexpression.
For example :
echo "blia blib bou blf" | sed 's/bl\(ia|f\)//g'
should return
" blib bou "
but it ...
- 1,113
91
votes
3
answers
280k
views
How to use 'sed' with piping
I want to replace a string outputted from grep, I have:
$ npm info webpack | grep version
it outputs me
$ version: '2.1.0-beta.12',
but I want to have:
$ 2.1.0-beta.12
So I think I might ...
- 1,035
66
votes
7
answers
69k
views
How to mass prepend text to file names?
Lets say I have a directory full of .md files all named various things. Lets say I wanted to prepend "test" to the front of each file name. So for example: file a.md, b.md, and c.md would ...
- 957
65
votes
7
answers
67k
views
Newlines in sed on Mac OS X
I find that \n doesn't work in sed under Mac OS X.
Specifically, say I want to break the words separated by a single space into lines:
# input
foo bar
I use,
echo "foo bar" | sed 's/ /\n/'
But the ...
- 2,825
54
votes
4
answers
69k
views
How to use sed to remove null bytes?
What is the sed incantation to remove null bytes from a file? I'm trying:
s/\000//g
but that is stripping out strings of zeroes.
s/\x00//g
seems to have no effect. I'm trying to do this in a sed ...
- 1,437
51
votes
11
answers
66k
views
sed: how to replace line if found or append to end of file if not found?
With a single input file that only contains comments (starting with #) and VARIABLE=value lines, is it possible to replace a value for a single variable if found and, otherwise, append the pair to the ...
- 1,258
33
votes
3
answers
163k
views
How to replace line in file with pattern with sed?
I'm reading a lot of documentation on sed, and am still stumped on my particular use case.
I want to replace this line in a conf file with my own line:
Replace this line: #maxmemory <bytes>
...
- 343
32
votes
12
answers
33k
views
Efficiently remove the last two lines of an extremely large text file
I have a very large file (~400 GB), and I need to remove the last 2 lines from it. I tried to use sed, but it ran for hours before I gave up. Is there a quick way of doing this, or am I stuck with ...
- 879
31
votes
18
answers
85k
views
List only the device names of all available network interfaces
I want to get a list of all available Network-Device Names on my Linux server. I figured that
ifconfig
would do the job, however ifconfig produces quite much output:
eth0 Link encap:Ethernet ...
- 593
27
votes
1
answer
51k
views
What does `-n` option in sed do?
This is the man page entry for -n:
-n
suppress automatic printing of pattern space
I notice that when not using -n for certain operations, each line is printed to stdout (and the requested ...
- 11.5k
23
votes
6
answers
74k
views
How to get the pid of a running process using a single command that parse the output of ps?
I am looking for a single line that does return the pid of a running process.
Currently I have:
ps -A -o pid,cmd|grep xxx|head -n 1
And this returns the fist pid, command. I need only the first ...
- 11.9k
22
votes
7
answers
29k
views
How to print last character of a file
In Unix, using a simple command like sed, is there a way to print the last character of a file?
suvitha
22
votes
3
answers
54k
views
Delete the first known character in a string with sed
How does one delete the first known character in a string with sed?
For example, say I want to delete the first character (which is @) in the string "@ABCDEFG1234"
I want sed to verify whether "@" ...
- 539
20
votes
2
answers
18k
views
Edit first line of large text file
I have a huge text file, far too big for the whole thing to be paged into memory. All I need to do with this text file is edit the first line (its a CSV file and I need to alter the titles).
Is there ...
- 1,871
20
votes
1
answer
32k
views
replacing only part of matched pattern in sed
For tens of thousands of lines with the same structure, i want to go (e.g.) from here:
abcd 12345 qwerty asdfg
...to here:
abcd 12345,qwerty asdfg
...with
sed 's/[0-9]\ [A-Z]/,/g'
I can match ...
- 303
19
votes
4
answers
58k
views
Using sed get substring between two double quotes
I have a file
xyz... rsync: "/home/path/to/file": Permission denied (13) rsync:
"/home/path/to/file1": Permission denied (13) rsync:
"/home/path/to/file2": Permission denied (13) rsync:
"/home/path/...
- 527
19
votes
2
answers
50k
views
replace nth occurence of string in each line of a text file
I have large text files with space delimited strings (2-5). The strings can contain "'" or "-". I'd like to replace say the second space with a pipe.
What's the best way to go?
Using sed I was ...
- 387
18
votes
7
answers
39k
views
How can I replace a newline with its escape sequence?
Using sed to make text that contains certain characters suitable for use in string literals is straightforward:
sed "s/\\\\/\\\\\\\\/g"
sed "s/\\\"/\\\\\\\"/g"
But how can I do something similar ...
- 1,125
18
votes
3
answers
21k
views
Finding and deleting lines from all files recursively
I have managed to grep the occurences of a keyword by using
grep "KeyWord" . -r -n -i -I
but, the occurences of the KeyWord are too large. I just want to delete all the lines containing that word.
...
- 659
18
votes
3
answers
26k
views
How can I delete U+200B (Zero-width space) using sed
I have a very large file that has zero-width spaces scattered throughout. It takes too long to open and edit using vi so I'd like to delete all instances of the character using sed. The problem is, ...
- 295
17
votes
2
answers
25k
views
Properly escaping forward slash in bash script for usage with sed
I'm trying to determine the size of the files that would be newly copied when syncing two folders by running rsync in dry mode and then summing up the sizes of the files listed in the output of rsync.
...
- 175
16
votes
1
answer
5k
views
Using sed to enable .htaccess files in Apache config
I am trying to automate the provisioning of an Ubuntu Vagrant box with a Bash script. All is well, but I am having difficulty with configuring Apache to allow overrides from .htaccess files. In my ...
- 6,262
15
votes
3
answers
19k
views
How can I make a table of contents for a markdown document with Python/AWK/SED?
I have the following markdown document:
Heading-a
==========
---text---
Heading-b
------------
--- text ---
Heading-c
----------
--- text---
Heading-d
=======
--- text----
Heading-e
---
...
...
- 5,794
14
votes
5
answers
19k
views
Add to every end of line in Notepad++
I have a long text file
gallery-dl -g -i w4b027.txt >
gallery-dl -g -i a4b028.txt >
gallery-dl -g -i b4b029.txt >
gallery-dl -g -i c4b030.txt >
gallery-dl -g -i d4b031.txt >
gallery-dl -...
- 354
14
votes
4
answers
3k
views
How to recursively replace characters with sed?
Is it possible to replace occurrences of a character sequence recursively without iterating over the same sequence again?
By performing a sed as in the following scenarios I can get the mentioned ...
- 283
14
votes
3
answers
23k
views
Better way to do "echo $x | sed ..." and "echo $x | grep ..."
I often find this in scripts (and, I have to admit, write it myself):
a=$(echo "$x" | sed "s/foo/bar/")
or
if echo "$x" | grep -q foo
then
...
fi
Consider "foo" to include some regex stuff.
I ...
- 4,500
13
votes
3
answers
30k
views
use sed to replace nbsp, 160, Hex 00a0, Octal 240, non-breaking space
I am having some problems with config files which have the non-breaking space chars in them.
How should I specify that character with sed so I can replace it with a space.
sed -n 's/ / /g'
...
- 13.7k
13
votes
2
answers
10k
views
sed weirdness, unmatched {
I'm trying to use the r (read command to append the contents of data2 to data.
I want to append the contents after the line containing 'two'.
I want to delete the line containing 'two'.
But I've hit ...
13
votes
3
answers
5k
views
Cygwin bash sed locks my files
When I change files in cygwin bash with the sed command, the file gets locked.
Reproduce:
Open cmd and cd to non-user directory (f.e. temp)
echo aaa > test.txt
Open in texteditor, add line, try ...
user43881
12
votes
3
answers
18k
views
How can I cat/print a file except the last 2 lines?
How can get I get the contents of a text file, except the last (for instance) 2 lines, with standard or GNU shell tools? (bash, tail, cat and so on.)
- 402
12
votes
2
answers
9k
views
How to use verbatim strings in sed?
E.g.,
sed 's/string/ /g' where string="a\c:ti]\']x""/\//:`~$%#^&"'
That is how to pass the string to sed literally rather than as an interpreted regex pattern?
I'm going to use this with ...
- 389
11
votes
2
answers
12k
views
Multiple search and replace actions in one large text file
I have a big text file (about 2GB). I want to do five search and replace actions on the same file, and would like to do this in one command. Normally I use vim, open the file, do one replace action, ...
- 7,812
10
votes
1
answer
2k
views
Replacing [0-9] with [A-J] not working with sed
I am working on a command that replaces all digits 0-9 with their corresponding letters in sed. I know I'm doing something wrong, but sed is not interpreting the replacement regex as anything but a ...
- 101
10
votes
2
answers
36k
views
Sed only print matched expression
How to make sed only print the matched expression?
I want to rewrite strings like "Battery 0: Charging, 44%, charging" to
"Battery: 44%". I tried the following:
sed -n '/\([0-9]*%\)/c Battery: \1'
...
- 287
10
votes
1
answer
4k
views
Where does Macports install GNU sed when I install coreutils port?
Where does Macports install the GNU sed when I install coreutils port? I want to use GNU sed instead of BSD sed on Mac OS X.
- 9,039
10
votes
1
answer
5k
views
Can GNU sed (for Windows) handle Unicode? If so, is it a code-page/locale issue, or a switch?
I've been using GNU SED on and off for a couple of years now. It spins me out a bit sometimes, but it does a good job... for single-byte char sets!
I now and then notice references to GNU SED being ...
- 3,043
9
votes
5
answers
16k
views
Multiple sed commands in Bash
I have a a file usernames and passwords in JSON format that I want to convert to process.
I have used sed in different commands to process it but what I would like to know is how to lump all three ...
- 205
9
votes
4
answers
58k
views
Print line X lines before/after found line
I want to search for a particular string in a certain file. If I found the string, I also want to print the line X lines before (or after) that line.
Can this with grep or awk, or do I need a ...
- 213
9
votes
2
answers
14k
views
How greedy is sed in matching patterns?
I know sed is greedy when matching patterns. But, how greedy is it?
Consider these examples.
$ echo 'foobar123' | sed 's/[0-9]*/(&)/'
()foobar123
$ echo 'foobar123' | sed 's/[0-9][0-9]*/(&)/'...
- 191
9
votes
2
answers
12k
views
Find and replace with exclude cases in sed
I am trying to use sed to add a string after a certain pattern but I want to exclude the case where the additional string is already present.
For example:
"text.txt"
Hello
Bar
Hello world!
If I ...
- 238
8
votes
4
answers
29k
views
sed: replace any number of occurrences of a certain pattern
Given this input:
"hell -- 'this -- world --is'-- beautiful' --thanks-- we'-- are-- here"
I want to replace every '--' in between single quotes with 'X-X-X' using sed. It should give the following ...
- 81
8
votes
2
answers
2k
views
Emptying HTML tags with sed
I want to empty the content of every HTML tag but "keeping the structure".
From:
<h5>Holdrs <div class="tooltip" data-tooltip="Accounts with ..."></div>&...
- 91
8
votes
1
answer
65k
views
sed: can't read : No such file or directory
I use this as str_replace for every file in folder after unzipng archive in temp folder
find "$tmp" -type f | xargs sed -i "s/${targetsubstring}/${newsubstring}/g"
but I get this error:
sed: can't ...
- 175
8
votes
5
answers
44k
views
sed: extracting value of a key-value pair in a URL query string
I am trying to use sed to extract the value part of one of the many key-value pairs in a URL's query string
This is what I am trying:
echo 'http://www.youtube.com/watch?v=abc&g=xyz' | sed 's@^...
- 583
8
votes
2
answers
40k
views
Using sed command for a specific line
I want to change a cell in file when my desired state exists. Example:
File:
id311 vmName1 state0
id312 vmName2 state0
id313 vmName3 state0
I will type a script and this script ...
- 805
8
votes
2
answers
26k
views
sed, insert file before last line
The following inserting file method for sed used to be working, even before last line, but not any more. Is it a bug in the current sed?
Demo of inserting file method with sed:
mkdir /tmp/test
...
- 8,821