Personal


microexcelanlyst

by George Taniwaki

While working toward my Microsoft Data Science Certificate (see Jul 2017 blog post), I also completed the Microsoft Excel for the Data Analyst XSeries Program sponsored by Microsoft and edX.

There are 3 classes in the program, two of which were also courses for the Microsoft Data Science Certificate.

DAT205x – Introduction to Data Analysis using Excel

This basic course on data analysis using Excel covers pivot tables, using SUMIF() and SUMIFS() functions to create dashboards and year-over-year comparison tables (something that is not possible using pivot tables alone), creating reports with hierarchal data, using Power Pivot, and creating multi-table reports using the data model and the More tables… feature.

Year-over-year comparison tables can also be created using the Excel data model and time intelligence functions. These are covered in the third course, DAT 206x.

Time: Since I am already an experienced Excel user, I skipped the videos and just did the homework. I covered the 8 modules in about 6 hours

Score:  I missed 1 quiz question and no lab questions for a combined score of 99%

image     image

DAT222x – Essential Statistics for Data Analysis using Excel

[I took DAT222x in order to earn the Microsoft Data Science Certificate. This section is copied from this  Jul 2017 blog post.]

This class is comprehensive and covers all the standard statistics and probability topics including descriptive statistics, Bayes rule, random variables, central limit theorem, sampling and confidence interval, and hypothesis testing. Most analysis is conducted using the Data analysis pack add-in for Excel.

Time: I used to work in market research, so I know my statistics. However, there are 36 homework assignments and it took me over 20 hours to complete the 5 modules.

Score: I missed 9 questions on the quizzes (88%) and six in the final exam (81%) for a combined score of 86%. (Despite the time it takes to complete, homework counts very little toward the final grade)

DAT222x-Score_thumb10     DAT222x-Certificate21

DAT206x – Analyzing and Visualizing Data with Excel

Topics include importing data and using queries with Excel, the Excel data model, using the M query language and DAX query language, creating dashboards and visualizations, and using Excel with Power BI.

Within the M language, topics include using the functions in the ribbon, including filtering rows, Table.Unpivot function.

Within the DAX language, topics include using the X functions like SUMX() and the CALCULATE() function, using Calendar table and time intelligence, and customize pivot tables and pivot charts using the CUBE functions from multidimensional expressions (MDX) language. The CUBE functions can also generate a table that can be used to create chart types that Excel does not support directly from a pivot table (for instance the new treemap, sunburst, and histogram charts).

Time: I am an experienced Excel user, but some of the advanced DAX functions were new to me. 6 hours for 8 modules

Score: I got a bit sloppy. I missed 2 lab questions and 1quiz question for a combined score of 95%

DAT206x Score     DAT206x Certificate

Final Certificate

Below is my certificate of completion for the Microsoft Excel for the Data Analyst XSeries Program.

ExcelXSeries Certificate

Advertisements

MsftBigData

by George Taniwaki

Big data and machine learning are all the rage now. Articles in the popular press inform us that anyone who can master the skills needed to turn giant piles of previously unexplored data into golden nuggets of business insight can write their own ticket to a fun and remunerative career (efinancialcareers May 2017).

Conversely, the press also tells us that if we don’t learn these skills a computer will take our job (USA Today Mar 2014). I will have a lot more to say about changes in employment and income during the industrial revolution in future blog posts.

But how do you learn to become a data scientist. And which software stack should one specialize in? There are many tools to choose from. Since I live in the Seattle area and do a lot of work for Microsoft, I decided to do take an online class developed and sponsored by Microsoft and edX. Completion of the course leads to a Microsoft Data Science Certificate.

The program consists of 10 courses with some choices, like conducting analysis using either Excel or Power BI, and programming using either R or Python. Other parts of the Microsoft stack you will learn include SQL Server for queries and Microsoft Azure Machine Learning (MAML) for analysis and visualization of results. The courses are priced about $99 each. You can audit them for free if you don’t care about the certificates.

I started the program in February and am about half way done. In case any clients or potential employers are interested in my credentials, my progress is shown below.

DAT101x – Data Science Orientation

If you haven’t been in college in a while or have never taken an online class, this is a good introduction to online learning. The homework consists of some simple statistics and visualization problems.

Time: 3 hours for 3 modules

Score: 100% on 3 assignments

DAT101x Score    DAT101x Certificate

DAT201x – Querying with Transact-SQL

I took a t-SQL class online at Bellevue College two years ago. Taking a class with a real teacher, even one you never meet, was a significantly better experience than a self-paced mooc. This course starts with the basics like select, subqueries, and variables. It also covers intermediate topics like programming, expressions, stored procedures, and error handling. I did my homework using both a local instance of SQL Server and on an Azure SQL database.

Time: 20 hours for 11 modules

Score: I missed one question in the homework and two in the final exam for a combined score of 94%

DAT201x Score     DAT201x Certificate

DAT207x – Analyzing and Visualizing Data with Power BI

I already have experience creating reports using Power BI. I also use Power Query (now called get and transform data) and M language and Power Pivot and DAX language, so this was an easy class.

The course covers data transforms, modeling, visualization, Power BI web service, organization packs, security and groups. It also touches on the developer API and building mobile apps.

Time: 12 hours for 9 modules

Score: I missed one lab question for a combined score of 98%

DAT207x Score     DAT207x Certificate

DAT222x – Essential Statistics for Data Analysis using Excel

This class is comprehensive and covers all the standard statistics and probability topics including descriptive statistics, Bayes rule, random variables, central limit theorem, sampling and confidence interval, and hypothesis testing. Most analysis is conducted using the Data analysis pack add-in for Excel.

Time: I used to work in market research, so I know my statistics. However, there are 36 homework assignments and it took me over 20 hours to complete the 5 modules.

Score: I missed 9 questions on the quizzes (88%) and six in the final exam (81%) for a combined score of 86%. (Despite the time it takes to complete, homework counts very little toward the final grade)

DAT222x Score     DAT222x Certificate

DAT204x – Introduction to R for Data Science

Now we are getting into the meat of the program. R is a functional language. In many ways it is similar to the M language used in Power Query. I was able to quickly learn the syntax and grasp the core concepts.

The course covers vectors, matrices, factors, lists, data frames, and simple graphics.

The lab assignments use DataCamp which has a script window where you write code and a console window that displays results. That makes it easy to debug programs as you write them.

The final exam used an unexpected format. It was timed and consisted of about 50 questions, mostly fill-in-the-blank responses that include code snippets. You are given 4 minutes per question. If you don’t answer within the time limit, it goes to the next question. I completed the test in about 70 minutes, but I ran out of time on several questions, and was exhausted at the end. I’m not convinced that a timed test is the best way to measure subject mastery by a beginning programmer. But maybe that is just rationalization on my part.

Time: 15 hours for 7 modules

Score: I got all the exercises (ungraded) and labs right and missed two questions in the quizzes. I only got 74% on the final, for a combined score of 88%

DAT204x Score     DAT204x Certificate

DAT203.1x Data Science Essentials

The first three modules in this course covered statistics and was mostly a repeat of the material introduced in DAT222x. But the rest of the course provides an excellent introduction to machine learning. You learn how to create a MAML instance, import a SQL query, manipulate it using R or Python, create a model, score it, publish it as a web service, and use the web service to append predictions as a column in Excel. I really like MAML. I will post a review of my experience in a future blog post.

The course was a little too cookbook-like for my taste. It consisted mostly of following directions to drag-drop boxes onto the canvas UI and copy-paste code snippets into the panels. However, if you want a quick introduction to machine learning without having to dig into the details of SQL, R, or Python, this is a great course.

Time: 10 hours for 6 modules

Score: 100% on the 6 labs and the final

DAT203.1x Score     DAT203.1x Certificate

I have now completed six out of the ten courses required for a certificate. I expect to finish the remaining 4 needed for a certificate by the end of the year. I will also probably take some of the other elective courses simply to learn more about Microsoft’s other machine learning and cloud services.

For my results in the remaining classes, see Microsoft Data Science Certificate-Part 2

Update: Modified the description of the final exam for DAT204x.

by George Taniwaki

LotteChocoPie

Moon pies for cheap. Photo by George Taniwaki

I love moon pies (apparently, I was a southerner in a past life). Surprisingly, they are big in South Korea too (who knew, for history see Wikipedia).

Incidentally, don’t confuse moon pies with moon cakes which are another Asian sweet (which I usually don’t like because of the salty egg flavor).

Anyway, today, I found a really cheap source of my favorite confection. Lotte brand is $3.50 for 335g or 29 cents a pie. Mysteriously, they are hidden next to weird spices in the international food aisle, not prominently displayed with the other cookies in the snack aisle. Perhaps it’s a form of American food protectionism by US cookie makers, Asian segregationist policy or redlining by the store, or the result of some other nativist conspiracy plot.

It’s crazy that a South Korean company can import all the ingredients, process them, ship them back to the U.S., and still be cheaper than US-made cookies. But I don’t care as long as I get my fix of graham cracker, marshmallow, and sugary goodness.

by George Taniwaki

I just returned from a short trip to New York. I have been to the city many times, but not recently. So I took time to go to places that are new since my last visit in 2009.

Cooper Hewitt Smithsonian Design Museum

CooperHewitt

Immersion room, courtesy Cooper Hewitt Smithsonian Design Museum

As a software program manager, the Cooper Hewitt is one of my favorite museums. It recently completed a major renovation (Press Release, Dec 2014). I was looking forward to seeing the redesigned design museum and was not disappointed.

Upon entering the museum, each visitor is given a stylus and a code number. The stylus is a bit bulky but is rugged. The pointed end can be used with large touchscreen monitors (probably Microsoft PixelSense devices since the newer Surface Hub wasn’t released until after the museum opened) scattered around some rooms. Visitors can select images, write text, and draw images on the touchscreens. Visitors can tap the other end of the stylus to the touchscreens to save their work. They can also tap on exhibit signs to save them and get more information for later.

On the second floor is a cubical Immersion Room that contains another large touchscreen monitor. On this one, visitors can select wallcovering patterns from the Cooper Hewitt collection or design their own using the pen. They can save their patterns and project them on the walls of the room. It is a very enjoyable experience to see your pattern fill the room (see photo above).

After your visit, you can go to the Cooper Hewitt website, create an account, enter your code, and review your visit and further explore exhibits that interested you. If you are a developer or tinkerer, check out the Toys section to use the API and to access anonymized visitor data.

Museum of Modern Art

moma_conner_crossroads197663

Crossroads (promotional still) 1976, Courtesy Connor Family Trust

The Museum of Modern Art is not new yet. However, since my last visit, MoMA has announced a major expansion. An increase of 4,600 sq. m (50,000 sq. ft) will add about 17%  of new space the the already large museum. Construction has started, though it hasn’t caused any closure of the current space for now.

The addition is expected to be well integrated with the existing museum. Construction will take over four years to complete (Curbed New York, Jan 2016).

I saw a special exhibit on Bruce Conner (1933-2008) an avant garde painter, sculptor, photographer, and film maker (see photo). The show was organized by the San Francisco Museum of Modern Art. In an ironic twist, I was unable to visit the newly remodeled SFMoMA while I was in SF in April since it was still closed for renovation (see SF Gate, May 2016).

The expansion of MoMA required the demolition of the American Folk Art Museum, which was a lovely bronze-clad building next door to it. (I saw a wonderful special exhibit on quilts during my last visit to New York.) The building is already gone and is now just a hole in the ground. The museum has moved to Columbus Avenue between 65th and 66th Streets.  I didn’t have time to visit it.

9/11 Memorial & Museum

911Memorial04

The north fountain with white rose. Photo by George Taniwaki

The National September 11 Memorial opened in 2014. It honors the victims killed in New York, Washington, DC, and Pennsylvania during of the awful attacks in September, 2001 as well as the people killed in the earlier attack on the World Trade Center in February, 1993.

The memorial consists of two square fountains, each encompassing the footprint of one of the towers. The fountains are surrounded by bronze panels with the names of each victim cut into them. A white rose is placed by each name on that person’s birthday (see photo). Water falls about 10 m (30 ft) into a reflecting pool. From there, it falls into a square hole so deep you cannot see the bottom. The sound of rushing water is emitted from the hole. The scale of the fountains is moving. Even on a busy summer weekend when thousands of tourists are viewing the memorial, there is plenty of space to stand and contemplate.

911MuseumBalcony08P

View from the balcony entering the museum. Photo by George Taniwaki

911MuseumBlueSky04P

View of quote from stairs entering the museum. Photo by George Taniwaki

The 9/11 museum is also enormous and also within the footprint of the towers. The main floor is under the fountain about 18m (60ft) below ground. From the balcony you can see the slurry wall that holds back the Hudson River and a giant remnant of a column from the World Trade Center marked with spray paint (see photo).

An escalator takes you past a quote translated from Virgil’s Aeneid, “No day shall erase you from the memory of time” (see photo). The quote is surrounded by 2,983 watercolor paintings, one for each victim, by Spencer Finch recalling the shade of blue of the sky on the morning of 9/11.

Overall, the museum does a good job of explaining the events leading to the attack and the recovery effort afterwards. One can imagine the difficult task of presenting an evenhanded account in the face of enormous pressure from victim families, first responders, government agencies, donors, and politicians. Nearly every artifact and photo is heavily researched and annotated, which is somewhat chilling. One gets a sense of how invested the survivors are in preserving the memories of the loved ones lost in the attacks.

As a side note, when viewed in context, the use of the Virgil quote above is rather controversial. The “you” refers to the attackers, not the victims, giving it a very different meaning than what was intended (NY Times, Apr 2011). Yet the alternate interpretation is also true. Because of the horror they caused, we will not forget the attackers either.

The Cloisters

MetCloisters

A garden in the Cloisters. Courtesy of Metropolitan Museum of Art

Hudson04P

Panoramic view of the Hudson River and New Jersey. Photo by George Taniwaki

The Cloisters is part of the Metropolitan Museum of Art. It houses artwork and architectural fragments from twelfth to fifteenth century Europe. The building is shaped like a medieval cloister and is located at the top of a hill in Fort Tryon Park in northern Manhattan (see photos).

I’m not a big fan of art from this era, so have never bothered to go all the way up to the Cloisters. But after visiting Florence a few years ago I decided I should make the journey. It was well worth it. I took the subway and walked up the hill. The building is imposing.  The gardens are tranquil and beautiful. And seeing the historical transition in painting from flat to perspective is fascinating.

Dyckman Farmhouse Museum

Dyckman

Irises in bloom. Courtesy of Dyckman Farmhouse Museum

On my way to the Cloisters, I accidentally missed my stop. Rather than wait for a train the other direction, I decided to walk back. On a busy street filled with businesses I suddenly passed by a small garden and walked in. It turns out it was part of the Dyckman Farmhouse Museum. Opened in 1916, the museum is not very large, but is well maintained and reveals some of the history of Manhattan beginning with the Revolutionary War.

Final Notes

When visiting a city, I rarely buy City Pass tickets or their equivalents. The list of venues is fixed and limited. Instead, I just buy tickets in advance directly from each museum I visit. In fact, I just go to each museum first, checked out how long the line is, and if it is too long, then use my phone to buy tickets online. Museums with long lines and that lack online ticket sales don’t get my business.

To get around a strange town, I need more than a map. I need info on the best ways to get around by walking, biking, using public transit, or hailing Uber. Also, I want to compare estimated travel times and costs for each option. I have found two excellent apps to help. They are CityMapper and Transit, both available for iPhone and Android.

by George Taniwaki

This is a continuation of my class notes from Landscaping in the Northwest without the Need for Automatic Sprinklers, taught by noted local plant writer and speaker Marianne Binetti and sponsored by the Cascade Water Alliance.

Part 1 of this blog entry is posted in April 2014.

LandscapeNote05

Plants mentioned on page 5, Gold band yucca, Miss Willmott’s ghost, Blue fescue, Allium

LandscapeNote06

Plants mentioned on page 6, Smoke tree, Golden ninebark, Huckleberry, Elderberry, Salmonberry

LandscapeNote07

Plants mentioned on page 7, Rosemary, Sedum razzleberry

LandscapeNote08

Plants mentioned on page 8, Sword fern, Sedum autumnjoy, Euphorbia

LandscapeNote09

Plants mentioned on page 9, Begonia, Hen and chicks, Sempervivum glabifolium, Sedum angelina, Woolly thyme, Sandwort, Moss lawn, Blue star creeper

LandscapeNote10

Plants mentioned on page 10, Sedum echeverias, Creeping jenny, Christmas rose

Note: The hyperlinks to nurseries and garden shops in this blog post were added by me and are for reference only. They were not part of the lecture and are not meant as endorsements by me or the instructor.

by George Taniwaki

Last Saturday, I attended a free class sponsored by the Cascade Water Alliance, an association of several water districts in Western Washington. The class, entitled Landscaping in the Northwest without the Need for Automatic Sprinklers, was held in Sammamish, an eastern suburb of Seattle. It was taught by noted local plant writer and speaker Marianne Binetti. It was a great class and I learned a lot. Probably the most important thing I learned was to not be intimidated by plant names. There are thousands of varieties of ornamental plants and you can’t know them all. Just go to the nursery with a plan to buy water-conserving plants, ask for help, and pick things you like.

MarrianeBinetti

Marianne Binetti. Photo by Joe B

Ironically, it is raining today. But I know that come summer, the Puget Sound region will have three dry months. Our house has a big yard and out in the suburbs water is very expensive. I want to learn how to design and build a low maintenance/low irrigation landscape for our yard. And you can’t beat the price of a free class.

My wife Sue and I attended a similar class back when we lived in Denver. That class featured a technique called xeriscaping and was taught by the Denver Water Department. Denver is much drier than Seattle (15-inches rain per year in Denver vs. 38 inches in Seattle). However, we had a city house with a tiny yard plus water was much cheaper due to the senior water rights held by the city.

Both the Denver low-water xeriscaping class and Saturday’s no-water class covered how to choose drought-tolerant and freeze-tolerant plants like succulents, hardy perennials, and prairie grasses. These plants are not necessarily native to the respective regions, but will grow there and look appealing. Both classes also offered information on proper soil amendment, and mulching to minimize or eliminate the need for irrigation.

Enough description about the classes. Below are the notes I took during Ms. Benetti’s lecture. I took the notes on a 6-inch square notebook I received from Northwest Kidney Centers using a pen with black ink. I cut the pages out of the book, added the colors using felt-tip highlighter pens, and scanned the pages. Enjoy!

LandscapeNote01

Plants mentioned on page 1, Heavenly bamboo, Japanese red pine

LandscapeNote02

Plants mentioned on page 2, Wisteria, Clematis

LandscapeNote03

Plants mentioned on page 3, Barberry, Euonymous, Lavender, Spiraea

LandscapeNote04

Plants mentioned on page 4, Kinnick kinnick, Rock rose, Potentilla

Part 2 of this blog entry is posted in April 2014.

Note: The hyperlinks to nurseries and garden shops in this blog post were added by me and are for reference only. They were not part of the lecture and are not meant as endorsements by me or the instructor.

by George Taniwaki

It was raining all day, keeping me from doing yard work this afternoon. That’s my excuse and I’m sticking to it. So I started doodling and came up with an invention to keep exterior concrete steps from getting covered with puddles of water.

My idea is to use a hard rubber mold on top of each tread to cut notches in the step that act as gutters to guide the water down to the next step down.

The drawing below shows the cross-section of a typical concrete step with a 11-inch tread and 7-inch riser. The tread has a series of 1-inch wide triangular grooves molded into it. The grooves are sloped down the tread. The groove is 1/8-inch deep at the start of the tread (cross-section AA) and reaches 1/2-inch deep (to become 45-degrees) at the end of the tread (cross-section BB).

The design has several nice features.

  1. Unlike most concrete steps, the treads are not sloped. The tops of the grooves are level to the ground. This improves safety. You won’t slide, and potentially fall, when the steps are wet, muddy, or icy
  2. The grooves shed water from the treads. This keeps water from forming puddles on the treads even if the concrete for the steps is poured slightly off-level
  3. The grooves reduces the contact area between the step and your shoes, to increase traction, which again improves safety

ConcreteSteps

Concrete steps design. Drawing by George Taniwaki

I’ve never seen concrete steps with grooves like this. So either I am the first person to think of this, or my design is a solution in search of a problem that doesn’t exist.

Next Page »