first_imgViruses have a bad connotation.  We immediately think of the ones that cause disease: “I’ve got a virus,” you say when feeling under the weather.  Actually, you have trillions of them all the time, even in the best of health.  A single gram of stool sample can have 10 billion of them!  What does that mean?  Scientists are only beginning to find out.    One thing it means is that they can’t be all bad.  Elizabeth Pennisi reported in Science this week about work at the University of British Columbia and Washington University to explore the human virome.1  She began her report,In the past decade, scientists have come to appreciate the vast bacterial world inside the human body.  They have learned that it plays a role in regulating the energy we take in from food, primes the immune system, and performs a variety of other functions that help maintain our health.  Now, researchers are gaining similar respect for the viruses we carry around.Bacteria have been easier to count than the tiny viruses.  Many of our internal viruses are bacteriophages that invade and kill bacteria.  This suggests they play a role in keeping the brakes on bacterial infections.  “For every bacterium in our body, there’s probably 100 phages,” Pennisi wrote.  The number of virus species identified in stool samples of healthy adults varied from 52 to 2773.  “The viromes varied significantly from one individual to the next; they were even more diverse than the bacterial communities within the same individuals,” Pennisi reported.  “But each person’s viral community remained stable over the course of the year.”  That is, unless they go on a different diet or eating regimen; then the viromes change.  But people who eat the same foods tend to converge on virus profiles.  Researchers also found that infants with fevers had more viruses than healthy infants.    We are full of viruses, in other words, but we don’t know what they all do.  This is “a true frontier” of research, with much to learn.  “Ultimately, those viruses are incredibly important in driving what’s going on” one scientist from the University of British Columbia said.  It’s not enough to know your bacteria; you have to know the viruses that interact with them. 1.  Elizabeth Pennisi, “Microbiology: Going Viral: Exploring the Role Of Viruses in Our Bodies,” Science, 25 March 2011: Vol. 331 no. 6024 p. 1513, DOI: 10.1126/science.331.6024.1513.It’s always been intriguing that viruses look incredibly well designed.  Some bacteriophages look like lunar landing capsules, legs and all.  Scientists have learned that some viruses have shells like hard plastic (05/07/2004) and pack their DNA into their capsids with motors generating remarkable force, in an orderly manner (03/20/2007, 12/30/2008).  They are also extremely effective in finding their target cells, inserting their DNA, and commandeering the genetic machinery to make copies of themselves.    Evolutionists don’t know what to do with viruses.  They are not considered transitional forms between molecules and life.  Intelligent design would describe their design and predict that they have functions, but would be at a loss to explain harmful viruses.  It takes Biblical creation to explain that they were probably designed for good originally, but some became harmful because of the Fall due to sin.  The analogy might be to a science fiction movie where robotic servants went berserk, or to the broom of the sorcerer’s apprentice that multiplied and could not be stopped.  Sometimes a single mutation can turn a beneficial bacterium into a disease-causing terror; the same could be true with viruses.    Maybe they were intended to be regulators of bacteria.  Maybe they were designed to convey information to the body about new environments, and were equipped to copy themselves to spread the word so that the body could be prepared.  Who knows?  This is, after all, a frontier of research.  For philosophers, it’s noteworthy that we are stumbling onto a reality right around us – right within us – about which we have been largely oblivious, with the potential to dramatically change our understanding of nature.    Given that an athlete running the high hurdles in the peak of health is carrying around trillions of viruses, intuition suggests that most of what they do for us is good.  The scientific research appears poised to find many beneficial functions for our viral passengers.  It happened with bacteria; it took society a long time to change the emotional response from “germs… uggh!” with the householder running to get the antibacterial spray, to an appreciation of the many good things bacteria do for us.  Now we look differently upon our bacterial passengers.  We have learned they outnumber our own cells, and are learning that our viral passengers outnumber the bacteria 100 to one.  Expect amazing things to be discovered about these tiny, mysterious machines.(Visited 8 times, 1 visits today)FacebookTwitterPinterestSave分享0last_img read more

first_imgViolent protests against Sanjay Leela Bhansali’s Padmaavat in Gujarat and Rajasthan, a day before the film’s release on Thursday, forced the Multiplex Association of India to decide against screening the period drama in the two States.Incidents of violence were also reported from towns and cities in Haryana, Maharashtra, Uttar Pradesh, Madhya Pradesh, Bihar and Jammu where protesters torched buses, blocked highways and created a ruckus outside theatres.The Gujarat Multiplex Owners’ Association announced that it would not run the controversial film in any theatre in the State in “view of the prevailing law and order situation.”“We have taken the decision for the safety of our customers as well as to save our property,” said association president Manubhai Patel.50 arrestedThe Ahmedabad police have arrested 50 people and filed three separate FIRs in connection with the violence witnessed at three malls in the city on Tuesday night.Hitting out at the State government, Patidar leader Hardik Patel sought to know if Chief Minister Vijay Rupani would resign in view of the situation.Protesters from Shri Rajput Karni Sena and other outfits went on the rampage in different parts of Rajasthan on Wednesday.The Jaipur-Delhi National Highway was blocked at several places.Protesters damaged two State roadways buses in Sikar district. No one was injured in the violence.Multiplex Association of India president Deepak Asher said its members will not screen Padmaavat in Rajasthan. The association has decided not to screen it in two other States — Goa and Madhya Pradesh — in view of the law and order situation.The historic Chittorgarh Fort, where Queen Padmini is believed to have lived in the 13th century, was closed for a day after women members of Karni Sena tried to break in for a protest.Karni Sena founder Lokendra Singh Kalvi said at a press conference in Jaipur that a “self-imposed curfew” would be enforced by the people to boycott the film.A tight security net was thrown around Diggi Palace in the State capital, where the Jaipur Literature Festival is starting on Thursday, in view of Karni Sena’s warning of a protest against the presence of Central Board of Film Certification chief Prasoon Joshi at the event.Members of Karni Sena targeted movie halls in Meerut, Saharanpur and Muzaffarnagar districts of Uttar Pradesh and blocked the Delhi-Saharanpur highway in protest against the screening of Padmaavat.last_img read more

first_imgGlobe Business launches leading cloud-enabled and hardware-agnostic conferencing platform in PH Don’t miss out on the latest news and information. Kevin Durant out with Achilles injury; to undergo MRI on Tuesday PLAY LIST 03:12Kevin Durant out with Achilles injury; to undergo MRI on Tuesday02:25Raptors or Warriors? PBA players take their pick of NBA champ01:43Who are Filipinos rooting for in the NBA Finals?01:29Police teams find crossbows, bows in HK university01:35Panelo suggests discounted SEA Games tickets for students02:49Robredo: True leaders perform well despite having ‘uninspiring’ boss02:42PH underwater hockey team aims to make waves in SEA Games01:44Philippines marks anniversary of massacre with calls for justice01:19Fire erupts in Barangay Tatalon in Quezon City The Warriors opened the third on a 16-9 burst to lead for the first time all night, then went on to score 100 or more points for a 14th straight game.Draymond Green had 12 points, six assists and five rebounds as the Warriors finished with 40 assists.Curry and Thompson shot a combined 4 for 16 in the first half, including 2 of 10 on 3s, before Curry found his stroke and wound up 9 for 19 overall in his 14th 30-point performance of the season. Thompson struggled to nine points on 4-for-10 shooting, missing all three of his 3-point tries.The Knicks made 11 of their initial 20 shots while the Warriors began 8 of 18 and 1 for 8 from behind the arc to fall behind 29-19 — Curry missed four of his first five field goals.Andre Iguodala, who had dealt with a bruised left calf that kept him out the previous two games, returned to the Warriors rotation and scored five points in 20 minutes off the bench.ALL KINDS OF ALL-STARSGreen and Thompson were named All-Star reserves, joining Curry — a captain — and Durant.The Warriors became the first team in NBA history to have four All-Stars in back-to-back seasons. It will be the ninth time in NBA history that a team has had four All-Stars play in the game.“For us to all go is pretty special,” Thompson said. “First time in NBA history to have four guys go two years in a row is incredible.”Golden State had the same four All-Stars a year ago.“I’m proud of them, amazing to have four All-Stars two years in a row,” Kerr said. “We’re obviously really lucky to have these guys all together at once on the roster. What I love is they’re so unselfish, they play together so well. That’s why they’re all All-Stars. It wouldn’t work if they were selfish and looking for their own shot, looking for their own points.”TIP-INS MOST READ Typhoon Kammuri accelerates, gains strength en route to PH 2 ‘newbie’ drug pushers fall in Lucena sting Golden State Warriors’ Stephen Curry, left, guards New York Knicks’ Jarrett Jack (55) during the first half of an NBA basketball game Tuesday, Jan. 23, 2018, in Oakland, Calif. (AP Photo/Ben Margot)OAKLAND, Calif. — Kevin Durant mouthed off to official James Williams, got tossed late and might have cost himself another assist during a career night of dishing out the ball.“I told him he was wrong,” Durant said afterward of arguing with Williams in the first half over a carry.ADVERTISEMENT NEXT BLOCK ASIA 2.0 introduces GURUS AWARDS to recognize and reward industry influencers Stephen Curry scored 17 of his 32 points in the third quarter to go with seven assists and six rebounds, and the Golden State Warriors used a big second half to beat the New York Knicks 123-112 on Tuesday night.Durant added 14 points, a career-high 14 assists and two blocks before his ejection with 2:50 left for a second technical. Zaza Pachulia added 13 points in Golden State’s seventh straight victory in the series.FEATURED STORIESSPORTSWATCH: Drones light up sky in final leg of SEA Games torch runSPORTSLillard, Anthony lead Blazers over ThunderSPORTSMalditas save PH from shutout“Second half, his whole thing was like he’s trying to get me. … He was searching for me, he’s looking to try to tech me up to get me back because he’s still got his feelings hurt from the first half,” Durant said. “That’s what’s been going on around the league the whole year, a bunch of that.“I gotta keep my head a little bit but I was upset. I’m a human being, too, I get upset,” Durant added. View commentscenter_img Knicks: F Kristaps Porzingis was held out with left knee irritation. … The Knicks dropped to 3-7 on the road vs. the Western Conference. … New York’s 14 turnovers led to 16 Golden State points.Warriors: Durant notched his 16th career double-double of points and assists. … Rookie Jordan Bell missed his second straight game with inflammation in his left ankle. … Golden State’s winning streak vs. the Knicks is its longest since a nine-game unbeaten run against New York from November 1963-November 1964. The Warriors have won 24 of the last 30 in the series.QUOTEABLEThree years after Thompson’s 37-point quarter on Jan. 23, 2015, against the Kings, Kerr recalled the moment.“It was almost surreal, the frenzy. It was the combination of Klay going crazy and licking his chops and his teammates and the joy that they had for him and with him,” Kerr said. “They were just hunting shots for him. It was like a pack of hyenas.”UP NEXTKnicks: At Denver on Thursday night continuing a seven-game road trip.Warriors: Host Timberwolves on Thursday night looking for a fourth straight home win in the series and third overall. Slow and steady hope for near-extinct Bangladesh tortoises Read Next Kammuri turning to super typhoon less likely but possible — Pagasa Brace for potentially devastating typhoon approaching PH – NDRRMC Kuzma gets 28, leads Lakers past rival Celtics John Lloyd Cruz a dashing guest at Vhong Navarro’s wedding LATEST STORIES Curry made 8 of 15 from deep while sporting sneakers featuring the faces of his daughters Riley and Ryan and the message “I can do all things.”“I had those in my closet for a while and it’s actually something with (Under Armour) you can obviously design your shoes so I did it a long time ago,” Curry said. “Just walked by ’em today and was like, ‘all right, let me go back and rock the Curry 1s with my daughters on the side to give me a little inspiration.’ It’s nice being home and all that kind of stuff.”The Warriors bounced back from Saturday’s eight-point defeat at Houston to improve to 10-0 following a loss this season.Michael Beasley scored 21 points and Courtney Lee contributed 20 as New York lost for the third time in four games and sixth in eight.“We got lucky a little bit in the first half,” Lee said. “They were missing a lot of opens shots, and then in the second half those shots that they were missing went down and we weren’t able to get stops and get out and convert on those opportunities.”ADVERTISEMENTlast_img read more

first_imgA nurse who preyed on elderly patients in her care told lawyers with an inquiry examining her actions that she wouldn’t have been able to murder eight people if more controls were in place on medication at long-term care homes.Elizabeth Wettlaufer’s musings on what could have prevented her crimes are contained in a transcript of a lengthy interview submitted as evidence in the public inquiry that got underway in St. Thomas, Ont., this week.The 50-year-old injected more than a dozen patients with overdoses of insulin while working in long-term care homes and private residences in Ontario for nearly a decade. Her crimes went undetected until she confessed them to mental health workers and police in 2016.“I’ve given a lot of thought to changes that could have been made where I would not have been able to do this,” Wettlaufer said in an interview conducted Feb. 14 at the prison where she’s serving a life sentence after pleading guilty to eight counts of first-degree murder, four counts of attempted murder and two counts of aggravated assault.“If there was a way that the insulin was counted, I would not have been able to do what I did without getting caught.”Insulin, Wettlaufer said, should be treated like a controlled substance.Wettlaufer said she first began thinking about killing people in 2006 when she was in a Woodstock, Ont., hospital where she had gone to deal with her obsessive and depressive thoughts. She was diagnosed with borderline personality disorder and was prescribed medication, she said.“I just had so much anger I couldn’t concentrate … but if I thought about killing somebody while I was talking to other people I could talk normally to them,” Wettlaufer said. “And that’s where the thoughts of killing somebody started.”Wettlaufer said she began working shortly thereafter at Caressant Care, a long-term care home in Woodstock, and soon became overwhelmed.She said she was often working double shifts from 3 p.m. until 7 a.m. the following day, in charge of 32 patients with little help and dealing with a stressful home life.One night in 2007, she said she noticed Clotilde Adriano, 87, was on insulin.“This thought came into my head, just give her too much insulin and see what happens,” Wettlaufer said. “So I did.”Adriano didn’t die, but it sparked something for Wettlaufer, she said. She did the same thing to Adriano’s sister, Albina Demedeiros, 90, who was in a room across the hall, she said.“I chose them based on the fact they couldn’t self-report because they had dementia,” she said. Wettlaufer pleaded guilty to two charges of aggravated assault in connection with the sisters.She would go on to kill James Silcox in 2007 with an overdose of insulin. He was her first murder.Wettlaufer told lawyers with the inquiry that she picked insulin because it wasn’t tracked in the home and it could be deadly.“There was never a time when it was questioned why were we at some times going through insulin more so than others,” she said.Sometimes another nurse would check the amount of insulin she had prepped to give a patient. But that was easily circumvented, she said.“Even though another nurse checks it you can change the amount at any time,” she said.There were two rooms where she could get insulin and they were behind doors without windows, she added.“If the treatment room and med room were completely made of glass … there’s no way I could have done what I did without somebody seeing me,” Wettlaufer said.Even if there were windows, she explained, she could slip a few insulin pens into her pocket. And the pens themselves had no mechanism to restrict the amount of insulin — doses can be dialled up or down easily, she said.“If there was, say, a stopgap measure on the pen that once you dialled up a certain amount you couldn’t dial any more, or even if pens came preloaded … there wouldn’t be excess insulin,” she said.The public inquiry, which is scheduled to last several months, has said it won’t compel Wettlaufer to testify as her confessions and interview have been submitted as evidence.last_img read more

first_imgAdvertisement Twitter Advertisement Login/Register With: Could the story of the Vegas Golden Knights be coming soon to a theatre near you? (Getty Images/CBC Sports) Advertisementcenter_img The Vegas Golden Knights’ run to the Stanley Cup Final feels like a ready-made sports movie — clichés and all.A scrappy group of literal outcasts defying the odds, sticking it to the establishment and rallying after a tragedy to reach improbable heights? The script practically writes itself.So, might as well cast it and start production, right? LEAVE A REPLY Cancel replyLog in to leave a comment The Golden Misfits — working title — has the potential to fuse the sheer lunacy of Slap Shot and emotional stakes of Miracle with the mass appeal of the Mighty Ducks film franchise. The NHL even brought up the subject with the Golden Knights recently, with the players and coaches weighing in on their cinematic counterparts.But in order to achieve peak authenticity, it’s time for an expansion draft (er, casting call) to find the right actors. And should Vegas win the Cup, it’d be a bigger heist than anything Danny Ocean ever schemed up… Facebooklast_img read more

first_imgEgypt is providing Indian companies “duty-free access” to the markets in the European Union, North American and other African and Arab markets. Around 52 Indian companies are working in Egypt with an investment to the tune of $3 billion (about Rs 20,400 crore).Last month, Egyptian ambassador to India Hatem Tageldin addressed a business meet in Kolkata; he noted that the bilateral trade between the two countries stood at approximately $5 billion (about Rs 34,000 crore) every year, the Economic Times reported. Tageldin asked the participating Indian businesses to take advantage of the current investment opportunities in Egypt. According to him, the climate of investment in the Arab-land has reportedly witnessed major reforms, which would have an impact on domestic investments as well as on foreign direct investment (FDI).At the same time, Egyptian companies are looking at making additional investments in India. Kapci coatings, an Egypt-based company, which makes auto refinished products, is planning to invest about $50 million in setting up five factories in India. Kapci has already begun constructing its first unit in southern state Karnataka, the ambassador was quoted as saying by the ET.The Egyptian government has also launched various projects to create investment opportunities in sectors such as roads, railways, water plants, ports, solar energy among others. Meanwhile, 25 Indian companies are expected to take part in the Cairo International Fair (CIF), which would be held from March 16 to March 25, 2016. Some of the Indian companies operating in Egypt are: Dabur Egypt Limited, Essel Proback Egypt, Kirloskar Egypt, Ranbaxy Egypt Limited, Gas Authority of India (GAIL), Monginis and Pidilite Industries Egypt.[1 lakh = 100,000 | 1 crore = 10 million | 100 crore = 1 billion]last_img read more

first_imgBigg Boss Marathi 2 eliminationColors MarathiBigg Boss Marathi 2 has become a pot of boiling water in just 2 weeks of its airing on television. From displaying bad behaviour to getting physical with each other, contestants have crossed all their limits. Among six of these nominated contestants – Veena Jagtap, Parag Kanhere, Abhijit Kelkar, Maithili Jaavkar, Madhav Devchke and Neha Shitole – host Mahesh Manjrekar announced at the Weekend Cha Daav episode that Parag was in the safe zone and Abhijit was in the danger zone.As we know now that Parag has been saved from this week’s elimination, it remains to be seen which of the other five contestants will get eliminated from the show.And it was Maithili Jaavkar who got eliminated tonight after getting the least votes from the viewers. Before elimination, Maithili told Manjrekar that she will feel bad if she gets evicted tonight as she didn’t get to perform or got a chance to actively participate in any given task.In tonight’s episode of Bigg Boss Marathi 2, Mahesh Manjrekar had asked Abhijeet Bichukale and Surekha Punekar to start a conversation in English to lighten up the ambience in the house.Veena and Shivani, who had gotten physical with each other during the weekly Chor Bazaar task, were made to sit on a chair. Manjrekar had asked other contestants some question and whoever among the two girls get maximum votes, will be given a cold water shower. Keep watching this space for latest updates on Bigg Boss Marathi season 2.last_img read more

first_imgFormer Awami League MP Golam Maula Roni speaks after joining BNP at its chairperson`s Gulshan office on Monday. Photo: Prothom AloFormer Awami League MP Golam Maula Rony formally joined BNP on Monday, reports UNB.He joined the party at a simple ceremony at BNP chairperson’s Gulshan office in the evening.BNP secretary general Mirza Fakhrul Islam Alamgir welcomed Rony to their party, saying their democratic movement will be intensified with his joining.Speaking on the occasion, Rony said he joined BNP consciously deserting his party Awami League.Later, he received the BNP nomination letter for contesting the general election from Patuakhali-3.last_img

first_imgJalpaiguri (WB): West Bengal Chief Minister Mamata Banerjee today stressed on a permanent solution for the problems plaguing the tea gardens in north Bengal and formed an expert committee to find out a solution. She said that the committee, under Chief Secretary Malay De, would submit its report after going through the problems at the tea gardens, thus help in finding out a solution. “I want a permanent solution for the tea gardens. Nobody has ever thought of any permanent solution regarding the problems, raging at the tea gardens, for ages. Even the Centre has failed to keep up to its promise, made prior to the last Assembly elections, about opening the closed tea gardens,” Banerjee told reporters here at the Uttarkanya. Also Read – Speeding Jaguar crashes into Merc, 2 B’deshi bystanders killed Today, she held a meeting with representatives of the tea gardens. She also held a meeting senior ministers such as state Labour Minister Moloy Ghatak, Arup Biswas and senior officials of her government. The state government also proposed a daily wage of Rs 176 for the tea garden workers pending finalisation of the minimum wages. This year the state government had announced two interim hikes of Rs 17.50 and of Rs nine, taking the daily wage to Rs 159. Meanwhile, the largest tea producing state of Assam, last week, had announced issuing a possible notification on an interim hike of Rs 30, with retrospective effect from March 1 this year, which would take the current daily wage of tea garden workers from Rs 137 to Rs 167. Banerjee also lambasted the culture of strike at the tea gardens and said it finished the work culture there.last_img read more

first_imgFebruary 5, 2018 This story originally appeared on PCMag Register Now » Free Webinar | Sept. 9: The Entrepreneur’s Playbook for Going Global If you haven’t seen it by now, Amazon’s Super Bowl LII commercial is fairly clever. Kudos to the company for coming up with a fun way to mash together celebrities and Alexa without it feeling overly cheesy or trying-too-hard. More importantly, a big thanks to Amazon’s engineers who came up with an ingenious way to broadcast the “A word” without it triggering everyone’s Echos — whatever version of the device you own.The most annoying thing about watching any YouTube video or televised commercial that mentions Alexa is that it typically triggers your Echo device to get ready to respond to a query. Or worse, the person you’re watching who says “Alexa” just keeps on babbling, which then makes your Echo do something you didn’t want it to do — or just apologize for being unable to do whatever commands it tried to interpret.And since Amazon definitely wanted its Super Bowl LII to mention she-who-shall-not-be-named, and refer to her frequently, the company had to come up with a different way to do so in order to avoid hacking off everyone who already owns an Echo device.The solution? Acoustic fingerprinting.”The trick is to suppress the unintentional waking of a device while not incorrectly rejecting the millions of people engaging with Alexa every day,” said Shiv Vitaladevuni, a senior manager on the Alexa Machine Learning team, in an Amazon blog post.Though Amazon isn’t detailing the specific techniques its using to keep your Echo from triggering from its Super Bowl advertising, Bloomberg notes that a Reddit user, Asphyhackr, might have figured out Amazon’s secret.”I did a little research tonight and found that the Echo, while it’s processing the wake word, searches the Audio Spectrum and if is significantly quieter in the area of 4000hz to 5000hz, she will not wake for the word,” Asphyhackr writes.”I found that when I analyzed the spectrum of them saying her name, the spectrums were significantly quieter in the range of 3000hz to 6000hz. In some of those recordings, those frequencies appeared to be non-existent. In others it appeared like the boosted the surrounding frequencies to make the Echo see a gap in the spectrum.”In other words, if your Echo (Best Price at Amazon) notices something strange happening in the audio spectrum, it realizes that it should ignore whatever is being said — like “Alexa.” And while this works well when Amazon has a planned announcement to make, like an advertisement, the company has to get a bit more creative when it can’t anticipate the large-scale broadcast of its digital helper’s name.”When multiple devices start waking up simultaneously from a broadcast event, similar audio is streaming to Alexa’s cloud services. An algorithm within Amazon’s cloud detects matching audio from distinct devices and prevents additional devices from responding. The dynamic fingerprinting isn’t perfect, but as many as 80 to 90 percent of devices won’t respond to these broadcasts thanks to the dynamic creation of the fingerprints,” reads Amazon’s blog. 3 min read Growing a business sometimes requires thinking outside the box.last_img read more

first_imgThe Iris dataset is the simplest, yet the most famous data analysis task in the ML space. In this article, you will build a solution for data analysis & classification task from an Iris dataset using Scala. This article is an excerpt taken from Modern Scala Projects written by Ilango Gurusamy. The following diagrams together help in understanding the different components of this project. That said, this pipeline involves training (fitting), transformation, and validation operations. More than one model is trained and the best model (or mapping function) is selected to give us an accurate approximation predicting the species of an Iris flower (based on measurements of those flowers): Project block diagram A breakdown of the project block diagram is as follows: Spark, which represents the Spark cluster and its ecosystem Training dataset Model Dataset attributes or feature measurements An inference process, that produces a prediction column The following diagram represents a more detailed description of the different phases in terms of the functions performed in each phase. Later we will come to visualize pipeline in terms of its constituent stages. For now, the diagram depicts four stages, starting with a data pre-processing phase, which is considered separate from the numbered phases deliberately. Think of the pipeline as a two-step process: A data cleansing phase, or pre-processing phase. An important phase that could include a subphase of Exploratory Data Analysis (EDA) (not explicitly depicted in the latter diagram). A data analysis phase that begins with Feature Extraction, followed by Model Fitting, and Model validation, all the way to deployment of an Uber pipeline JAR into Spark: Pipeline diagram Referring to the preceding diagram, the first implementation objective is to set up Spark inside an SBT project. An SBT project is a self-contained application, which we can run on the command line to predict Iris labels. In the SBT project,  dependencies are specified in a build.sbt file and our application code will create its  own SparkSession and SparkContext. So that brings us to a listing of implementation objectives and these are as follows: Get the Iris dataset from the UCI Machine Learning Repository Conduct preliminary EDA in the Spark shell Create a new Scala project in IntelliJ, and carry out all implementation steps, until the evaluation of the Random Forest classifier Deploy the application to your local Spark cluster Step 1# Getting the Iris dataset from the UCI Machine Learning Repository Head over to the UCI Machine Learning Repository website at https://archive.ics.uci.edu/ml/datasets/iris and click on Download: Data Folder. Extract this folder someplace convenient and copy over iris.csv into the root of your project folder. You may refer back to the project overview for an in-depth description of the Iris dataset. We depict the contents of the iris.csv file here, as follows: A snapshot of the Iris dataset with 150 sets You may recall that the iris.csv file is a 150-row file, with comma-separated values. Now that we have the dataset, the first step will be performing EDA on it. The Iris dataset is multivariate, meaning there is more than one (independent) variable, so we will carry out a basic multivariate EDA on it. But we need DataFrame to let us do that. How we create a dataframe as a prelude to EDA is the goal of the next section. Step 2# Preliminary EDA Before we get down to building the SBT pipeline project, we will conduct a preliminary EDA in spark-shell. The plan is to derive a dataframe out of the dataset and then calculate basic statistics on it. We have three tasks at hand for spark-shell: Fire up spark-shell Load the iris.csv file and build DataFrame Calculate the statistics We will then port that code over to a Scala file inside our SBT project. That said, let’s get down to loading the iris.csv file (inputting the data source) before eventually building DataFrame. Step 3# Creating an SBT project Lay out your SBT project in a folder of your choice and name it IrisPipeline or any name that makes sense to you. This will hold all of our files needed to implement and run the pipeline on the Iris dataset. The structure of our SBT project looks like the following: Project structure We will list dependencies in the build.sbt file. This is going to be an SBT project. Hence, we will bring in the following key libraries: Spark Core Spark MLlib Spark SQL The following screenshot illustrates the build.sbt file: The build.sbt file with Spark dependencies The build.sbt file referenced in the preceding snapshot is readily available for you in the book’s download bundle. Drill down to the folder Chapter01 code under ModernScalaProjects_Code and copy the folder over to a convenient location on your computer. Drop the iris.csv file that you downloaded in Step 1 – getting the Iris dataset from the UCI Machine Learning Repository into the root folder of our new SBT project. Refer to the earlier screenshot that depicts the updated project structure with the iris.csv file inside of it. Step 4# Creating Scala files in SBT project Step 4 is broken down into the following steps: Create the Scala file iris.scala in the com.packt.modern.chapter1 package. Up until now, we relied on SparkSession and SparkContext, which spark-shell gave us. This time around, we need to create SparkSession, which will, in turn, give us SparkContext. What follows is how the code is laid out in the iris.scala file. In iris.scala, after the package statement, place the following import statements: import org.apache.spark.sql.SparkSession Create SparkSession inside a trait, which we shall call IrisWrapper: lazy val session: SparkSession = SparkSession.builder().getOrCreate() Just one SparkSession is made available to all classes extending from IrisWrapper. Create val to hold the iris.csv file path: val dataSetPath = “>\\iris.csv” Create a method to build DataFrame. This method takes in the complete path to the Iris dataset path as String and returns DataFrame: def buildDataFrame(dataSet: String): DataFrame = {/* The following is an example of a dataSet parameter string: “C:\\Your\\Path\\To\\iris.csv”*/ Import the DataFrame class by updating the previous import statement for SparkSession: import org.apache.spark.sql.{DataFrame, SparkSession} Create a nested function inside buildDataFrame to process the raw dataset. Name this function getRows. getRows which takes no parameters but returns Array[(Vector, String)]. The textFile method on the SparkContext variable processes the iris.csv into RDD[String]: val result1: Array[String] = session.sparkContext.textFile(>) The resulting RDD contains two partitions. Each partition, in turn, contains rows of strings separated by a newline character, ‘\n’. Each row in the RDD represents its original counterpart in the raw data. In the next step, we will attempt several data transformation steps. We start by applying a flatMap operation over the RDD, culminating in the DataFrame creation. DataFrame is a view over Dataset, which happens to the fundamental data abstraction unit in the Spark 2.0 line. Step 5# Preprocessing, data transformation, and DataFrame creation We will get started by invoking flatMap, by passing a function block to it, and successive transformations listed as follows, eventually resulting in Array[(org.apache.spark.ml.linalg.Vector, String)]. A vector represents a row of feature measurements. The Scala code to give us Array[(org.apache.spark.ml.linalg.Vector, String)] is as follows: //Each line in the RDD is a row in the Dataset represented by a String, which we can ‘split’ along the new //line characterval result2: RDD[String] = result1.flatMap { partition => partition.split(“\n”).toList } //the second transformation operation involves a split inside of each line in the dataset where there is a //comma separating each element of that line val result3: RDD[Array[String]] = result2.map(_.split(“,”)) Next, drop the header column, but not before doing a collection that returns an Array[Array[String]]: val result4: Array[Array[String]] = result3.collect.drop(1) The header column is gone; now import the Vectors class: import org.apache.spark.ml.linalg.Vectors Now, transform Array[Array[String]] into Array[(Vector, String)]: val result5 = result4.map(row => (Vectors.dense(row(1).toDouble, row(2).toDouble, row(3).toDouble, row(4).toDouble),row(5))) Step 6# Creating, training, and testing data Now, let’s split our dataset in two by providing a random seed: val splitDataSet: Array[org.apache.spark.sql.Dataset[org.apache.spark.sql.Row]] = dataSet.randomSplit(Array(0.85, 0.15), 98765L) Now our new splitDataset contains two datasets: Train dataset: A dataset containing Array[(Vector, iris-species-label-column: String)] Test dataset: A dataset containing Array[(Vector, iris-species-label-column: String)] Confirm that the new dataset is of size 2: splitDataset.sizeres48: Int = 2 Assign the training dataset to a variable, trainSet: val trainDataSet = splitDataSet(0)trainSet: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [iris-features-column: vector, iris-species-label-column: string] Assign the testing dataset to a variable, testSet: val testDataSet = splitDataSet(1)testSet: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [iris-features-column: vector, iris-species-label-column: string] Count the number of rows in the training dataset: trainSet.countres12: Long = 14 Count the number of rows in the testing dataset: testSet.countres9: Long = 136 There are 150 rows in all. Step 7# Creating a Random Forest classifier In reference to Step 5 – DataFrame Creation. This DataFrame ‘dataFrame’ contains column names that corresponds to the columns present in the DataFrame produced in that step The first step to create a classifier is to  pass into it (hyper) parameters. A fairly comprehensive list of parameters look like this: From ‘dataFrame’ we need the Features column name – iris-features-column From ‘dataFrame’ we also need the Indexed label column name – iris-species-label-column The sqrt setting for featureSubsetStrategy Number of features to be considered per split (we have 150 observations and four features that will make our max_features value 2) Impurity settings—values can be gini and entropy Number of trees to train (since the number of trees is greater than one, we set a tree maximum depth), which is a number equal to the number of nodes The required minimum number of feature measurements (sampled observations), also known as the minimum instances per node Look at the IrisPipeline.scala file for values of each of these parameters. But this time, we will employ an exhaustive grid search-based model selection process based on combinations of parameters, where parameter value ranges are specified. Create a randomForestClassifier instance. Set the features and featureSubsetStrategy: val randomForestClassifier = new RandomForestClassifier() .setFeaturesCol(irisFeatures_CategoryOrSpecies_IndexedLabel._1) .setFeatureSubsetStrategy(“sqrt”) Start building Pipeline, which has two stages, Indexer and Classifier: val irisPipeline = new Pipeline().setStages(Array[PipelineStage](indexer) ++ Array[PipelineStage](randomForestClassifier)) Next, set the hyperparameter num_trees (number of trees) on the classifier to 15, a Max_Depth parameter, and an impurity with two possible values of gini and entropy. Build a parameter grid with all three hyperparameters: val finalParamGrid: Array[ParamMap] = gridBuilder3.build() Step 8# Training the Random Forest classifier Next, we want to split our training set into a validation set and a training set: val validatedTestResults: DataFrame = new TrainValidationSplit() On this variable, set Seed, set EstimatorParamMaps, set Estimator with irisPipeline, and set a training ratio to 0.8: val validatedTestResults: DataFrame = new TrainValidationSplit().setSeed(1234567L).setEstimator(irisPipeline) Finally, do a fit and a transform with our training dataset and testing dataset. Great! Now the classifier is trained. In the next step, we will apply this classifier to testing the data. Step 9# Applying the Random Forest classifier to test data The purpose of our validation set is to be able to make a choice between models. We want an evaluation metric and hyperparameter tuning. We will now create an instance of a validation estimator called TrainValidationSplit, which will split the training set into a validation set and a training set: val validatedTestResults.setEvaluator(new MulticlassClassificationEvaluator()) Next, we fit this estimator over the training dataset to produce a model and a transformer that we will use to transform our testing dataset. Finally, we perform a validation for hyperparameter tuning by applying an evaluator for a metric. The new ValidatedTestResults DataFrame should look something like this: ——–+ |iris-features-column|iris-species-column|label| rawPrediction| probability|prediction| +——————–+——————-+—–+——————–+ | [4.4,3.2,1.3,0.2]| Iris-setosa| 0.0| [40.0,0.0,0.0]| [1.0,0.0,0.0]| 0.0| | [5.4,3.9,1.3,0.4]| Iris-setosa| 0.0| [40.0,0.0,0.0]| [1.0,0.0,0.0]| 0.0| | [5.4,3.9,1.7,0.4]| Iris-setosa| 0.0| [40.0,0.0,0.0]| [1.0,0.0,0.0]| 0.0| Let’s return a new dataset by passing in column expressions for prediction and label: val validatedTestResultsDataset:DataFrame = validatedTestResults.select(“prediction”, “label”) In the line of code, we produced a new DataFrame with two columns: An input label A predicted label, which is compared with its corresponding value in the input label column That brings us to the next step, an evaluation step. We want to know how well our model performed. That is the goal of the next step. Step 10# Evaluate Random Forest classifier In this section, we will test the accuracy of the model. We want to know how well our model performed. Any ML process is incomplete without an evaluation of the classifier. That said, we perform an evaluation as a two-step process: Evaluate the model output Pass in three hyperparameters: val modelOutputAccuracy: Double = new MulticlassClassificationEvaluator() Set the label column, a metric name, the prediction column label, and invoke evaluation with the validatedTestResults dataset. Note the accuracy of the model output results on the testing dataset from the modelOutputAccuracy variable. The other metrics to evaluate are how close the predicted label value in the ‘predicted’ column is to the actual label value in the (indexed) label column. Next, we want to extract the metrics: val multiClassMetrics = new MulticlassMetrics(validatedRDD2) Our pipeline produced predictions. As with any prediction, we need to have a healthy degree of skepticism. Naturally, we want a sense of how our engineered prediction process performed. The algorithm did all the heavy lifting for us in this regard. That said, everything we did in this step was done for the purpose of evaluation. Who is being evaluated here or what evaluation is worth reiterating? That said, we wanted to know how close the predicted values were compared to the actual label value. To obtain that knowledge, we decided to use the MulticlassMetrics class to evaluate metrics that will give us a measure of the performance of the model via two methods: Accuracy Weighted precision val accuracyMetrics = (multiClassMetrics.accuracy, multiClassMetrics.weightedPrecision) val accuracy = accuracyMetrics._1 val weightedPrecsion = accuracyMetrics._2 These metrics represent evaluation results for our classifier or classification model. In the next step, we will run the application as a packaged SBT application. Step 11# Running the pipeline as an SBT application At the root of your project folder, issue the sbt console command, and in the Scala shell, import the IrisPipeline object and then invoke the main method of IrisPipeline with the argument iris: sbt consolescala>import com.packt.modern.chapter1.IrisPipelineIrisPipeline.main(Array(“iris”)Accuracy (precision) is 0.9285714285714286 Weighted Precision is: 0.9428571428571428 Step 12# Packaging the application In the root folder of your SBT application, run: sbt package When SBT is done packaging, the Uber JAR can be deployed into our cluster, using spark-submit, but since we are in standalone deploy mode, it will be deployed into [local]: The application JAR file The package command created a JAR file that is available under the target folder. In the next section, we will deploy the application into Spark. Step 13# Submitting the pipeline application to Spark local At the root of the application folder, issue the spark-submit command with the class and JAR file path arguments, respectively. If everything went well, the application does the following: Loads up the data. Performs EDA. Creates training, testing, and validation datasets. Creates a Random Forest classifier model. Trains the model. Tests the accuracy of the model. This is the most important part—the ML classification task. To accomplish this, we apply our trained Random Forest classifier model to the test dataset. This dataset consists of Iris flower data of so far not seen by the model. Unseen data is nothing but Iris flowers picked in the wild. Applying the model to the test dataset results in a prediction about the species of an unseen (new) flower. The last part is where the pipeline runs an evaluation process, which essentially is about checking if the model reports the correct species. Lastly, pipeline reports back on how important a certain feature of the Iris flower turned out to be. As a matter of fact, the petal width turns out to be more important than the sepal width in carrying out the classification task. Thus we implemented an ML workflow or an ML pipeline. The pipeline combined several stages of data analysis into one workflow. We started by loading the data and from there on, we created training and test data, preprocessed the dataset, trained the RandomForestClassifier model, applied the Random Forest classifier to test data, evaluated the classifier, and computed a process that demonstrated the importance of each feature in the classification. If you’ve enjoyed reading this post visit the book, Modern Scala Projects to build efficient data science projects that fulfill your software requirements. Read Next Deep Learning Algorithms: How to classify Irises using multi-layer perceptrons Introducing Android 9 Pie, filled with machine learning and baked-in UI features Paper in Two minutes: A novel method for resource efficient image classificationlast_img read more

first_imgTags: Alitalia, WTF Alitalia has gone bankrupt but at least they’re getting snazzy new uniforms << Previous PostNext Post >> Source: The Associated Press MILAN — Never underestimate the power of a new pair of shoes, or a brand new wardrobe for that matter.The bankrupt Italian airline, Alitalia, has commissioned the Alberta Ferretii fashion house to design new uniforms, the timing of which has raised a few eyebrows. The crew restyling comes as the Italian government evaluates binding offers for all, or part, of the airline with the intent of completing a sale by spring.Perhaps the airline is putting their best foot forward (in designer pumps, no less) to shake the stigma of bankruptcy?Alitalia has said the uniform update won¹t create any new financial burdens, and that it came at the request of cabin and ground crews for ‘greater comfort’.Ferretti has pledged to design a collection combining elegance with convenience to ensure that all personnel are “comfortable in each working environment and throughout any season.”The airline last updated its uniforms in 2016 with a burgundy-colored retro look for the first new uniforms in two decades.More news:  Venice to ban cruise ships from city centre starting next monthAlitalia said the new uniforms should be ready by summer. Share Thursday, November 30, 2017 last_img read more