Lines & ListsLesson 359Free preview

Find Duplicate Lines

This lesson is part of Web Developer Tools · All levels

Summary

Find Duplicate Lines searches for lines that appear more than once and usually displays the number of appearances. This tool helps find double entries to lists, customer data, or log. Since uniq only detects adjacent rows, the data needs to be sorted first before counting.

Key points

Duplicate is a line with the same content that appears at least twice
Sort followed by uniq -c showing the number of appearances of each line
Uniq -d only displays duplicate lines
The difference between capitalization and space can break the count
Use the results to clean up the data, not to delete it without checking.

Full lesson

Code examples and quizzes

Full lessons, code examples, and quizzes are available to subscribers. 5 questions.