Hi
Is there a way to remove duplicate records while composing a CSV?
Best Answer
A
Amanda Wong
said
almost 5 years ago
Hi Anandit,
If you have an array of items e.g. ["apple","apple","apple","pear","banana"], you can use formula .uniq to extract unique values from the array i.e ["apple","apple","apple","pear","banana"].uniq returns ["apple","pear","banana"]. This is a simple example that you may expand on.
To transform the array to a list data pill for use in a source list field, you can create a list using a combination of the following actions
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Append item to list action nested in Repeat action, create a new list, adding each unique item in each iteration --> this creates a new list data pill you can map in the source list field when composing csv
Screenshots of Recipe input:
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Workato List app, Append item to list action nested in Repeat action, add each unique item in each iteration --> this creates a new list data pill you can map in the source list field
4) Mapping source list field with the list data pill created in (3)
Answer
Amanda Wong
said
almost 5 years ago
Hi Anandit,
If you have an array of items e.g. ["apple","apple","apple","pear","banana"], you can use formula .uniq to extract unique values from the array i.e ["apple","apple","apple","pear","banana"].uniq returns ["apple","pear","banana"]. This is a simple example that you may expand on.
To transform the array to a list data pill for use in a source list field, you can create a list using a combination of the following actions
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Append item to list action nested in Repeat action, create a new list, adding each unique item in each iteration --> this creates a new list data pill you can map in the source list field when composing csv
Screenshots of Recipe input:
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Workato List app, Append item to list action nested in Repeat action, add each unique item in each iteration --> this creates a new list data pill you can map in the source list field
4) Mapping source list field with the list data pill created in (3)
L
Leanne Snoeck
said
over 3 years ago
Hi,
I am trying to remove duplicates from a CSV file based on IDs in a column, but the values are not organized as an array (they are separate values in a column). Is there a way to remove duplicates from the list? Example below
Input:
ID #
1
1
2
3
3
3
4
5
Desired Output:
1
2
3
4
5
Amanda Wong
said
over 3 years ago
Hi Leanne,
You can convert your csv into a workato array, then use formula to filter the array for unique rows.
Anandit Joshi
Removing Dupes from CSV
Hi Anandit,
If you have an array of items e.g. ["apple","apple","apple","pear","banana"], you can use formula .uniq to extract unique values from the array i.e ["apple","apple","apple","pear","banana"].uniq returns ["apple","pear","banana"]. This is a simple example that you may expand on.
To transform the array to a list data pill for use in a source list field, you can create a list using a combination of the following actions
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Append item to list action nested in Repeat action, create a new list, adding each unique item in each iteration --> this creates a new list data pill you can map in the source list field when composing csv
Screenshots of Recipe input:
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Workato List app, Append item to list action nested in Repeat action, add each unique item in each iteration --> this creates a new list data pill you can map in the source list field
4) Mapping source list field with the list data pill created in (3)
Amanda Wong
Hi Anandit,
If you have an array of items e.g. ["apple","apple","apple","pear","banana"], you can use formula .uniq to extract unique values from the array i.e ["apple","apple","apple","pear","banana"].uniq returns ["apple","pear","banana"]. This is a simple example that you may expand on.
To transform the array to a list data pill for use in a source list field, you can create a list using a combination of the following actions
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Append item to list action nested in Repeat action, create a new list, adding each unique item in each iteration --> this creates a new list data pill you can map in the source list field when composing csv
Screenshots of Recipe input:
1) Using Workato List app, create a list of size equal to no. of unique items in array
2) Using Repeat action, iterate through list in (1) i.e. iterate 3 times
3) Using Workato List app, Append item to list action nested in Repeat action, add each unique item in each iteration --> this creates a new list data pill you can map in the source list field
4) Mapping source list field with the list data pill created in (3)
Leanne Snoeck
Hi,
I am trying to remove duplicates from a CSV file based on IDs in a column, but the values are not organized as an array (they are separate values in a column). Is there a way to remove duplicates from the list? Example below
Input:
ID #
1
1
2
3
3
3
4
5
Desired Output:
1
2
3
4
5
Amanda Wong
Hi Leanne,
You can convert your csv into a workato array, then use formula to filter the array for unique rows.
See this recipe example: https://www.workato.com/recipes/1030990#recipe