"Append with Union" building block

BalazsBarany

Another user posted a very helpful building block for "appending" collections:

Handling collections: behavior of union vs append

 

If you get a "collection" result with each element having the same attributes, you can use the Append operator to "flatten" it to one example set. 

 

However, sometimes the attributes are not completely identical. This building block can work with this kind of data, as it uses the Union operator which creates a superset of the attributes.

 

A very useful building block, thanks to user @aruberutou!

JEdward

Thanks for pointing this out @BalazsBarany and thanks for creating it @aruberutou!

 

I've added an updated version of to the original post which scales nicely on larger datasets which you both might find useful. 

Let me know what you think. 

 

LAFAURIEA

Hello 

I'm a very beginner in Rapidminer, and try to manipulate some of my datas... trials and error type of stuff

 

After a loop attribute, I've got a collection of example set and try your "building block" on simple datas, the result is not what I'm looking for.

 

here is the capture.

it adds attribute(id) which is ok, but in this case it adds same datas of the others attribute at the end; I'd like only 14 lines

 

thanks

 

 

 

 

BalazsBarany

If you only want the 14 original lines, do an Aggregate operation with the original data as the grouping attributes and the max() or avg() of the new columns.

 

Why are you calculating the (standard) deviation of the id variable? It's numeric but the numbers don't have actual meaning.

LAFAURIEA

I just took an example for training so no real signification of the new attributes, just a try:

I took tutorial process of the loop attribute

then I store the collection and try merge this collection into a single table using your block. the problem is it gives a table with new colums for each attribute coming from each example set, that's OK but it replicates twice the others parameters (so in that case 2 times the same 14 examples with "?" and I just the 14 examples and get rid of the "?" like in picture attached in my previous post

 

 

BalazsBarany

Did you try the Aggregate route? 

 

We should take this discussion into the Studio area anyway; the output you're getting is exactly what you're supposed to get from "Append with Union". 

781194025

I'm having trouble with this! It looked like a miracle worker at first but now...

Even though the ID attribute is the same it simply creates 2 examples differing only by the Union attributes!

I want it to combine by ID damnit!

Here's the problematic output:

2366 iPhone suicide ruffles Chinese Apple supplier ? ? ? ? ? ? ? ? ? ? ? /Neutral/Surprise Neutral Chinese Apple supplier, iPhone suicide http://www.cbc.ca/news/world/iphone-suicide-ruffles-chinese-apple-supplier-1.855355 ? ? ? An employee at a factory that makes iPhones in China has killed himself after a prototype went missing, and Apple Inc. responded Wednesday by saying its suppliers are required to treat workers with dignity and respect. The dead worker, Sun Danyong, 25, worked in product communications at Foxconn Technology Group, a Taiwanese firm that makes many Apple items at a massive factory in the southern city of Shenzhen, near Hong Kong. Although Apple and Foxconn have confirmed Sun's suicide, they have not confirmed details about the death's circumstances that were reported by the state-run Southern Metropolis Daily, one of the region's most popular papers. Apple is a constant target of prying journalists, rabidly faithful customers and competitors who make great efforts to try to steal a peek at its latest gadgets. Sun was responsible for sending iPhone prototypes to Apple, and on July 13 he reported that he was missing one of the 16 units in his possession, the newspaper reported. His friends said company security guards searched his apartment, detained him and beat him, the paper reported. In the early morning of July 16, Sun jumped from the 12th floor of his apartment building, the paper said. Security chief denies beatings Jill Tan, an Apple spokeswoman in Hong Kong, would only issue a brief statement about the incident. "We are saddened by the tragic loss of this young employee, and we are awaiting results of the investigations into his death," Tan said. "We require our suppliers to treat all workers with dignity and respect." Foxconn said in a statement that its security chief has been suspended and turned over to the police. But the security official, Gu Qinming, was quoted by the Southern Metropolis Daily as saying he never hit Sun. Gu reportedly said that after three security personnel searched Sun's apartment and did not find the phone, the employee was ordered to go to Gu's office on July 15. The security chief said he didn't think Sun was being truthful about the phone, the paper reported. "I got a bit agitated. I pointed my finger at him and said that he was trying to shift the blame," Gu was quoted as saying. He added, "I was a little angry and I pulled his right shoulder once to get him to tell me what happened. It [the beating] couldn't have happened." Local police declined to respond to questions from The Associated Press. Foxconn executive Li Jinming said in a statement that Sun's death showed that the company needed to do a better job helping its employees with psychological pressures. "Sun Danyong graduated from a good school. He joined the company in 2008. He had an extremely bright future. The group and I feel deep pain and regret when a young person dies like this." © The Associated Press, 2009 Report Typo or Error Send Feedback Stay Connected with CBC News Mobile Facebook Podcasts Twitter Alerts Newsletter ? The Associated Press Posted: Jul 22, 2009 5:22 AM ET Last Updated: Jul 22, 2009 5:14 AM ET ? ? ? ? ? Surprise N 100.0 AGREEMENT OBJECTIVE NONIRONIC neg 0.0
4104 iPhone suicide ruffles Chinese Apple supplier ? ? ? ? ? ? ? ? ? ? ? /Neutral/Surprise Neutral Chinese Apple supplier, iPhone suicide http://www.cbc.ca/news/world/iphone-suicide-ruffles-chinese-apple-supplier-1.855355 ? ? ? An employee at a factory that makes iPhones in China has killed himself after a prototype went missing, and Apple Inc. responded Wednesday by saying its suppliers are required to treat workers with dignity and respect. The dead worker, Sun Danyong, 25, worked in product communications at Foxconn Technology Group, a Taiwanese firm that makes many Apple items at a massive factory in the southern city of Shenzhen, near Hong Kong. Although Apple and Foxconn have confirmed Sun's suicide, they have not confirmed details about the death's circumstances that were reported by the state-run Southern Metropolis Daily, one of the region's most popular papers. Apple is a constant target of prying journalists, rabidly faithful customers and competitors who make great efforts to try to steal a peek at its latest gadgets. Sun was responsible for sending iPhone prototypes to Apple, and on July 13 he reported that he was missing one of the 16 units in his possession, the newspaper reported. His friends said company security guards searched his apartment, detained him and beat him, the paper reported. In the early morning of July 16, Sun jumped from the 12th floor of his apartment building, the paper said. Security chief denies beatings Jill Tan, an Apple spokeswoman in Hong Kong, would only issue a brief statement about the incident. "We are saddened by the tragic loss of this young employee, and we are awaiting results of the investigations into his death," Tan said. "We require our suppliers to treat all workers with dignity and respect." Foxconn said in a statement that its security chief has been suspended and turned over to the police. But the security official, Gu Qinming, was quoted by the Southern Metropolis Daily as saying he never hit Sun. Gu reportedly said that after three security personnel searched Sun's apartment and did not find the phone, the employee was ordered to go to Gu's office on July 15. The security chief said he didn't think Sun was being truthful about the phone, the paper reported. "I got a bit agitated. I pointed my finger at him and said that he was trying to shift the blame," Gu was quoted as saying. He added, "I was a little angry and I pulled his right shoulder once to get him to tell me what happened. It [the beating] couldn't have happened." Local police declined to respond to questions from The Associated Press. Foxconn executive Li Jinming said in a statement that Sun's death showed that the company needed to do a better job helping its employees with psychological pressures. "Sun Danyong graduated from a good school. He joined the company in 2008. He had an extremely bright future. The group and I feel deep pain and regret when a young person dies like this." © The Associated Press, 2009 Report Typo or Error Send Feedback Stay Connected with CBC News Mobile Facebook Podcasts Twitter Alerts Newsletter ? The Associated Press Posted: Jul 22, 2009 5:22 AM ET Last Updated: Jul 22, 2009 5:14 AM ET ? ? ? ? ? Surprise N 100.0 AGREEMENT OBJECTIVE NONIRONIC ? ?

Why doesn't it put the Sentiment info in both? or merge them properly?

781194025

Why doesn't it merge these two examples when they only differ by the Union attribute Sentiment and SentimentScore? Every other attribute is the same including ID!!!

JEdward

As with the previous poster, you want either Aggregate or Join.  Union Append is not what you are looking for. @781194025