The Feed
An ongoing log of machine learning related notes, conversations, and thoughts from a consistent stream of podcasts, books, and articles.
Tuesday Nov 7th
Monday Nov 6th
Impressed with Deepsense.ai team — super active on kaggle
My own v1 deep learning landscape, trying to piece to together.
Sunday Oct 22nd
Building the Products Your Customers Need — Alluvium
Company focus:
- 1. We help them understand stability
- 2. Perfect production
Software is there to help, support, and enable experts in the all parts of the system.
Under the hood, fundamentally about dimensionality reduction…folks at the edge face high dimensionality problems day after day. Started with online learning off streaming data for the edge….but faced cultural and organizational friction.
Risk of adoption stalled conversations and POCs. Turned to, hey, what happens if we evaluate your historical data and immediately provide value/build longterm trust.
Historical event analysis — grounding it in real problem (stability score + real sensor data). Using shallow NN (lstm), unsupervised.
Saturday Oct 21st
Listening primarily to Andrew Ng.
- Today, the money is in supervised learning, structured data, and ads. He’s spoken before about the revenue Baidu was seeing (even a couple years ago) using deep learning for ad targeting, and while it’s not the sexiest use case, it’s certainly a prudent one.
- Ng said he thinks of building a business around AI as a multi-year “chess game,” where the goal is to strategically gather the best data, build models, build the product, and then learn even more from user data — ultimately creating a virtuous cycle.
- True AI company has these components.
— ‘strategic data acquisition’
— ‘centralized data warehouse’
— ‘pervasive automation’
— ‘new job descriptions'
Merlon with Bradford. Similar to skymind with financial industry fraud — crimes analysis.
- wanted to be here because of how unsexy, real biz problem it is
- catalysts: execs loosing jobs, amount of spending, amount of fines
- humans in the loop and ML product to help signal if someone opening a bank account should be doing that, notifying staff of users in the news for — potentially bad developments
- companies spending $10B+ each year on this problem
- risk ranking vs interest ranking (comps to his own startup data crawling, learn to rank implementations)
- provide a UI, full stack product
- generate some of their own data
- what is defensible. Product that ties within prop data, prop ui. Learning cross competitors.
- People/HR side is defensible too
Bradford speaking with Matt Turck.
- Largely the same pitch around vertical AI and why MLaaS/horizontal has — limited potential
- Focus on solving higher level business problem as opposed to low level task that will become commoditized pretty quickly
- Revenue
- “There is need to help people understand how machine learning can solve their problems and if you don’t come to them with a solution…but you kind of come with a really great piece of technology that could solve their solution…your ultimately drop the problem back in their lap.”
https://www.youtube.com/watch?v=mYu2MnPnCCY
New (to me) commentary on ideal team structures:
- Mostly see polar (very technical team…7 top kids from stanford) or (some guy from trucking business, domain expertise but no clue around tech)
- You see very little of very smart ML founder and smart business founder that realize they need to work together and also get PM, designer, subject matter expert, etc
- Sometimes you can overweight the domain expertise in first 1–2…startup commercial sense and savvy is more important in the beginning
Friday Oct 20th
Command C + Command V for the future tech IPOs. S-1
Thursday Oct 19th
Pod: https://overcast.fm/+HzyVPL4iU — Alluvium
Notes:
Manufacturing, broadly speaking, falls into two buckets.
- Discrete — monorail (car + plane assembly)
- Process/Additive — focus for Alluvium
- Producing a substance or material (refining oil, chemical processing), this has highly interdependent systems.
- Think of a visual aid…thousands of pipes and boilers wrangled together
Who the user is falls into two groups:
- someone working in oil refinery…typically relying on traditional threshold based results (alarm going off)
- team of individuals looking back at historical (corporate or incident resolution)
Product goal is to consolidate that and provide collective intelligence — system level analysis in real time…humans in the loop looking at feedback. + — labels
Not necessarily considered anomaly detection (thresholds have already been hit), rather — series of smaller changes and heads up
Technical under the hood:
- time series data of different frequency
- asset emitting state or sensor sharing temperature
- used unsupervised or semi machine learning (user)
- reality is most folks aren’t in a position to have that data..need to start from scratch and start injecting human expertise into the loop
- traditional ML (forecasting) and shallow NN (fluffy answer here, wonder if most folks worry about being perceived as not having deep tech)
- built deep empathy and humility for operators (most team lacked industrial DNA and domain knowledge)
General fears of a solution of this type
Corporate execs: fear that these tools that don’t actually know the problem and fear of how do we retain domain knowledge amongst workers that are aging / retire
- looking ways at software can build institutional knowledge
Front line side: machines are coming to replace job
- stress and messaging that system has to work alongside the human
Wednesday Oct 18th
and Part 2. Very well received by /r/machinelearing
- Neuron — thing that holds a number (between 0–1)…function
- Learning: computer finding right weights and biases
- Relu replaced sigmoid
- Cost function high when not training well, low when performant
- local minimum: doable
- global minimum: very hard
- find downhill direction and steepest descent
- a network learning is minimizing a cost function using gradient descent
Tuesday Oct 17th
Conversation with software engineer spending last year seriously exploring the field.
Draw to DL:
- Potential impact over the longterm
- Can play a role in bringing software eng practices to ml workflows
- Find ML problems to be more interesting than APIs/backend plumbing
- Business reality that folks are using ML right now
Shallow and new field
~1 yr of in-depth study can bring you to the edge of the field.
Commentary on the state of tooling being completely lacking…state of javascript 10 years ago
- model training, deployment, tracking, reproducibility
Monday Oct 16th
Podcast
Notes:
- We have a lot of resources, this is a big field that we’re well poised to play in.
- A lot of corporate speak oddly concluded with a candid and genuine comment that they don’t know of how some (non-silicon) business models will develop
Found this helpful in trying to understand automatic speech recognition vs natural language processing.
ASR — attempt to determine what, exactly, was said
- also known as speech to text
NLP —tries to determine the intent of what was said
Saturday October 14th
Post from Erik Bernhardsson who I’ve come to really respect from afar for discussions about ML, data, and org building.
This piece does a nice job of setting the scene. MOAR DATA =/ right or useful
Post.
A tweet from a month prior
again…real candid and grounded thoughts from someone who has actually built these systems. valuable to get this perspective.
Anecdotally found his reasoning as to why he took the CTO position to be one of the better I’m sharing a development in my professional life kind of updates.
Friday October 13th
Applied computer vision
- Special focus in object detection
- Matt is former longtime MSFT researcher. SEA → Beijing office
- 2012 mainstream debut / breakthrough for DL on-stage…NYT front page press
- ‘I still think it’s the early days of DL’
— Supervised is a bit more mature, semi/weakly semi supervision is needed
Applications
- Textiles Quality control (much higher interval testing vs random human sampling) material classification, fiber classification, and color analysis
— acquired data via human labels (microscope hooked up to inspection)
- Object detection within x-rays for safety within transportation
- Fashion — style/cust/other attributes
- Ecommerce
— dealing with billions of images/products on marketplaces and auction sites
— semi supervised learning is needed to scale with this volume…models trained via this.
- Semi-supervised learning is our key technical innovation
- “semi-supervised learning = subset of your data which is labeled with reliable quality and then using that to inform the rest of the data set”
- “weakly, in our definition….when you essentially have labels for all of your data but it’s unreliable (unbalanced, noisy) so that’s where actually find the biggest value…because the internet is full of it.”
- Scalability…moving away from dependence on human labeling
- China landscape is a bit different in perception. Far more open that folks think. Everyone is sharing notes. Rising tide overall. Information reaches the east but research (published in mandarin) in china doesn’t flow back through to US. Andrew Ng highlighted this on same pod.
Thursday October 12th
Box movements in ML space the past year.
Box has now done technical integrations / biz partnerships with
- IBM (text, audio)
- MSFT (video+audio)
- Goog (mostly image)
tl;dr
~ Machine learning as a service technology is effective and robust enough
~ Customers can see value in better searching, discovering, organizing enriched media. Far more efficient than humans (or no interaction at all)
~ Box can leverage best in class APIs…simply build out the plumbing on top of content management platform.
~ Said APIs are commoditized and become plug+play modular. Box here is an aggregator (HT Ben T) with end user relationship (billing, legal, UX, sales force).
Good Q&A with Aaron this summer.
Post
The more content you have and the more you use the system, the harder it gets to use. It’s counter-intuitive that this should keep progressing this way. The way to solve that problem, that scale, is through machine learning
Wednesday October 11th
Skymind — deep learning for the enterprise…an org and CTO I’ve been very intrigued by.
Notes from variety of sources.
- Podcast on Archillect.
- Tech talk, talk 2
- HN and /r/ML
Business:
- Largely serving China (healthcare) and Japan (banking, tech)
-US is more stable but conservative line of business
- 20 distributed people.
- Long sales in suits…working with systems integrators to scale selling to these large organizations.
- Tech agnostic…go to where data and dollars are. JVM systems.
- Open-core
Applications and solutions:
- Fraud detection (credit cards, online banking)
- Detecting illegal mining activity via satellite
- Network intrusion (packet analysis)
- Predicting machine failure
- Stock market trend prediction (up/down)
- Simbox fraud
- Finding similar parts for large vehicles
DL can be good at time series data, good at finding patterns in behavior
Problems largely revolving around analytics, churn prediction, recommendation engines
Technology:
- Created DL4J
- Main thing is production deployment software and a service level agreement for models we build.
- Not necessarily always DL, whatever the problem calls for
- Have built a lot of the software stack
Tuesday October 10th
https://www.youtube.com/watch?v=IMoQPvXMkJw
Building robust machine learning systems
- Ravelin — Protect merchants from credit card fraud
- 1–3% of revenue…cost for the merchant
- Serious precision/recall tradeoff. FP as a consumer (just checking, we think there may be fraud…False Neg — you don’t spot the fraud)
- I think most engineers would be horrified at the state of machine learning tooling (training, testing, deploying)
- ML systems
— High complexity
— Current decisions impact future performance
— Loss function doesn’t represent true goal
- Training models
— How do we make sure the world we present to the model is the view of the world we actually want to learn?
- Labels & truth
— Delayed labels….takes many weeks/months to acocunt for so if you just train on last 2–3mos…you actually don’t have that captured view of labels.
- Testing
— pull of set of examples you should *always* get right — use as unit test
— generate synthetic data for worst case scenarios (loading up credit cards in 10min, spending 100k)
Deploying
- Model provenance
— who trained the model…when…what git hash….on what data…which hyperparameters…..train/test performance
— Shipping model…share as much code as possible between offline and online
— Release gradually. A/B test.
— Score in live with both models. Inspect biggest disagreements.
Automation:
- Unfortunately end up training on *data_1_final_edited_really_final(1)(2).csv*
Monday October 9th
#AI methods ranked by econ value
Supervised Learning
Transfer Learning
Unsupervised Learning
Reinforcement Learning
Older notes to be backfilled
