Data validation for machine learning

Data validation for machine learning Breck et al., SysML’19

Last time out we looked at continuous integration testing of machine learning models, but arguably even more important than the model is the data. Garbage in, garbage out.

In this paper we focus on the problem of validation the input data fed to ML pipelines. The importance of this problem is hard to overstate, especially for production pipelines. Irrespective of the ML algorithms used, data errors can adversely affect the quality of the generated model.

Breck et al. describe for us the data validation pipeline deployed in production at Google, “used by hundreds of product teams to continuously monitor and validate several petabytes of production data per day.” That’s trillions of training and serving examples per day, across more than 700 machine learning pipelines. More than enough to have accumulated some hard-won experience on what can go wrong and the kinds of safeguards it is useful to have in place!

What could possibly go wrong?

The motivating example is based on an actual production outage at Google, and demonstrates a couple of the trickier issues: feedback loops caused by training on corrupted data, and distance between data Continue reading

Technology’s Promise – Highlights from DEF CON China 1.0

Technology's Promise - Highlights from DEF CON China 1.0
Technology's Promise - Highlights from DEF CON China 1.0

DEF CON is one of the largest and oldest security conferences in the world. Last year, it launched a beta event in China in hopes of bringing the local security communities closer together. This year, the organizer made things official by introducing DEF CON China 1.0 with a promise to build a forum for China where everyone can gather, connect, and grow together.

Themed "Technology's Promise", DEF CON China kicked off on 5/30 in Beijing and attracted participants of all ages. Watching young participants test, play and tinker with new technologies with such curiosity and excitement absolutely warmed our hearts!

It was a pleasure to participate in DEF CON China 1.0 this year and connect with local communities. Great synergy as we exchanged ideas and learnings on cybersecurity topics. Did I mention we also spoiled ourselves with the warm hospitality, wonderful food, live music, and amazing crowd while in Beijing.

Technology's Promise - Highlights from DEF CON China 1.0
Event Highlights: Cloudflare Team Meets with DEF CON China Visitors and Organizers (DEF CON Founder Jeff Moss and Baidu Security General Manager Jefferey Ma)

科技点燃未来，未来尽在指掌之间 — Cloudflare 与你共赏安全界 “奥斯卡” DEF CON China 1.0 大会

科技点燃未来，未来尽在指掌之间 — Cloudflare 与你共赏安全界 “奥斯卡” DEF CON China 1.0 大会
科技点燃未来，未来尽在指掌之间 — Cloudflare 与你共赏安全界 “奥斯卡” DEF CON China 1.0 大会

科技在发展，时代在进步，许多事情或许本质并没有改变，但呈现的方式已经日新月异，这或许就是我们常说的 — 未来。就像许多年前，我们还通过明信片和相册向亲友分享我们生活中的点点滴滴。许多年后，我们有了朋友圈、微博、Facebook、Instagram、抖音、各式博客。幼时还守着电视看着预录好的节目，接触外界的形形色色，现在我们透过直播的镜头，弹指间便能瞬息感受世界当下的脉动。

科技改变着我们，我们推动着未来

许多人都在电影中看到过极客指尖敲动，在数字的世界中急速驰骋的场景。然而现实生活中，这些人在哪儿不得而知。随着技术的发展，越来越多的年轻人加入了这个群体。在国外一直都有 DEF CON 这样的世界极客盛会。中国此前也还没有，直到去年 DEF CON 来到了中国，主办方斥巨资引进大会，想打造属于中国的技术社区，通过这样一个契机，将大家聚在一起，一同成长，最终构建一个属于中国自己的、真正的安全社区。于是，在 DEF CON 的名下，多了一个 DEF CON China。  

今年，DEF CON 经过一年的沉淀后，进入了正式版本 1.0，这个世界顶级的安全会议，在五月底，以 "Technology's Promise" — “科技点燃未来” 为主旨，于北京拉开了序幕，像是一位家长等待着 “孩子们” 一起过节。这个六一，还有什么能比来 DEF CON China 1.0 众乐乐更具意涵呢？

作为在中国地区的正式版本，DEF CON China 吸引了很多大咖前来参与，一直致力于网络安全的 Cloudflare，这次也前来共襄盛举，带来了最新的科技跟大家分享。

科技点燃未来，未来尽在指掌之间 — Cloudflare 与你共赏安全界 “奥斯卡” DEF CON China 1.0 大会
大会实况：Cloudflare 团队与DEF CON China 与会者和主办方进行交流 (DEF CON 创办人 Jeff Moss 与百度安全部总经理马杰)

六一儿童节前夕，小小参与者对新科技的好奇及探索新知识的向往令人对未来充满信心。(图源: Abhinav SP | #BugZee, DEFCON China )

DEF CON China 1.0 的徽章由 DEF CON 著名徽章设计师 Joe Grand 设计，采用柔性电路板打造，赋予冰冷的朋克气质艺术美感，用此激活点亮互动式艺术装置 “无极之树”。

科技点燃未来，未来尽在指掌之间 — Cloudflare 与你共赏安全界 “奥斯卡” DEF CON China 1.0 大会
Capture The Flag (CTF) 夺旗赛起源于DEF CON，是目前代表全球最高技术水平和影响力的 CTF。夺旗的赢家除了获得荣耀，也肩负一份责任，将极客精神传承并发扬光大。

Cloudflare 的使命是建立一个更好的互联网

Cloudflare 成立于 2009 年，是一家跨国科技公司，在全球 80 个国家部有 180 个数据中心。我们的性能和安全服务协同工作，以减少网站、移动应用程序和端到端 API 的延迟，同时防御 DDoS 攻击、滥用机器人和数据泄露。

此次大会是 Cloudflare 在区域深耕的第一小步。相信随着时间的推移，越来越多的用户会认识并了解 Cloudflare，此而加入我们。点此启用免费帐户，立即体验更快更安全的网络：cloudflare.com/sign-up

人才招聘中

Cloudflare 具有全球视野、本地化洞见的团队期待构建更好的全球互联网未来。我们北京和全球的办公室都在招聘人才，欢迎有志一同的伙伴加入我们！cloudflare.com/careers

科技点燃未来，未来尽在指掌之间 — Cloudflare 与你共赏安全界 “奥斯卡” DEF CON China 1.0 大会
Cloudflare 北京、新加坡、旧金山團隊齐聚一堂

Data center workloads become more complex despite promises to the contrary

Data centers are becoming more complex and still run the majority of workloads despite the promises of simplicity of deployment through automation and hyperconverged infrastructure (HCI), not to mention how the cloud was supposed to take over workloads.That’s the finding of the Uptime Institute's latest annual global data center survey (registration required). The majority of IT loads still run on enterprise data centers even in the face of cloud adoption, putting pressure on administrators to have to manage workloads across the hybrid infrastructure.To read this article in full, please click here

Data center workloads become more complex despite promises to the contrary

Data centers are becoming more complex and still run the majority of workloads despite the promises of simplicity of deployment through automation and hyperconverged infrastructure (HCI), not to mention how the cloud was supposed to take over workloads.That’s the finding of the Uptime Institute's latest annual global data center survey (registration required). The majority of IT loads still run on enterprise data centers even in the face of cloud adoption, putting pressure on administrators to have to manage workloads across the hybrid infrastructure.To read this article in full, please click here

Sponsored Post: Etleap, PerfOps, InMemory.Net, Triplebyte, Stream, Scalyr

Who's Hiring? 


  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services

  • For heads of IT/Engineering responsible for building an analytics infrastructure, Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. It automates most ETL setup and maintenance work, and simplifies the rest into 10-minute tasks that analysts can own. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • PerfOps is a data platform that digests real-time performance data for CDN and DNS providers as measured by real users worldwide. Leverage this data across your monitoring efforts and integrate with PerfOps’ other tools such as Alerts, Health Monitors and FlexBalancer – a smart approach to load balancing. FlexBalancer makes it easy to manage traffic between multiple CDN providers, API’s, Databases or any custom endpoint helping you achieve better performance, ensure the Continue reading

5G will augment Wi-Fi, not replace it

There’s arguably no technology topic that’s currently hotter than 5G. It was a major theme of the most recent Mobile World Congress show and has reared its head in other events such as Enterprise Connect and almost every vendor event I attend.Some vendors have positioned 5G as a panacea to all network problems and predict it will eradicate all other forms of networking. Views like that are obviously extreme, but I do believe that 5G will have an impact on the networking industry and is something that network engineers should be aware of.To read this article in full, please click here

Video chat about NRE Labs

Yesterday, I participated in a screen-cast with Derick Winkworth, aka @CloudToad, to discuss my blog posts about installing NRE Labs Antidote network emulator on your PC and creating lessons for NRE Labs. We also covered some general points like contributing to communities, how to get started blogging about technical topics, and more. Check it out, below:

This video, and other NRE Labs videos are available on YouTube. Also, the NRE Labs team runs a live screen-cast every Monday at 1:00 PM using the Discord app. Join the NRE Labs Discord channel and engage in the discussion.

Orange Matter: All I Want For Christmas is RESTCONF

Orange Matter Logo

I’ve been blogging for Solarwinds recently, posting on Orange Matter, with a cross-post to the Thwack Geek Speak forum. I need clear commands, structured response data and simple access to it all; so how about giving me REST-based APIs on all my infrastructure equipment?

This post appeared on Orange Matter as “All I Want For Christmas is RESTCONF“, but I’m also linking to the version posted on Thwack, in case you prefer to read and comment there.

I’d love it if you were to take a moment to visit and read, and maybe even comment!

(Featured image created by Kira auf der Heide on Unsplash)

If you liked this post, please do click through to the source at Orange Matter: All I Want For Christmas is RESTCONF and give me a share/like. Thank you!

Heavy Networking 453: BlueCat Networks Sinks DNS Claws Into Automation (Sponsored)

On today's Heavy Networking podcast we examine the role of DNS in infrastructure automation with sponsor BlueCat Networks. We look at how BlueCat hooks into third-party operational tools and get details on new integrations with Cisco DNA and ACI.

The post Heavy Networking 453: BlueCat Networks Sinks DNS Claws Into Automation (Sponsored) appeared first on Packet Pushers.

BrandPost: Moving to the Cloud? SD-WAN Matters! Part 2

This is the second installment of the blog series exploring how enterprises can realize the full transformation promise of the cloud by shifting to a business first networking model powered by a business-driven SD-WAN. The first installment explored automating secure IPsec connectivity and intelligently steering traffic to cloud providers. We also framed the direct correlation between moving to the cloud and adopting an SD-WAN. In this blog, we will expand upon several additional challenges that can be addressed with a business-driven SD-WAN when embracing the cloud:To read this article in full, please click here

BrandPost: It’s Time to Modernize Your Support Metrics

The good news is, there’s a lot of data coming into support organizations today through the many technologies and channels available. The bad news is all of this data can lead to metric overload. In the ever-changing customer support ecosystem, it can be difficult for managers to know which metrics really matter and where to focus for the best return.Deep breaths.It’s a good idea to periodically take a step back, look at the support metrics you’re tracking, and consider some newer metrics that may more accurately reflect today’s support center. Measuring the Customer Experience (CX) Typically, customer satisfaction metrics like Net Promoter Score and CSAT are used as a proxy for measuring CX. After all, it stands to reason that if customers are happy, they’re having a good experience. While it’s absolutely important to measure customer satisfaction, it’s equally as important to dig deeper into metrics that more accurately measure CX, especially as organizations increasingly focus on CX as a competitive differentiator.To read this article in full, please click here

1 … 1,217 1,218 1,219 1,220 1,221 … 3,841