From “Zombie Data” to “Smart Reports”

October 4, 2021 § Leave a comment

The bane of my IT existence is a business user who says, “Please get me the latest version of <random Excel file I have never seen before, named using idiosyncratic or ambiguous words>. Oh, and I need it tomorrow or else we won’t {make our numbers | pass our audit | satisfy the board}.”

I call this “zombie data” because it:

  • Lacks any self-awareness
  • Doesn’t remember where it came from
  • Has no relationship to its current context
  • Infects everyone it touches with that same mindlessness.
« Read the rest of this entry »

The Reporting Control Center

September 5, 2021 § 1 Comment

aka Quilt Data Hub or Lightdash 2.0?

Challenge

Can I evangelize
a corporate data platform
by just emailing out reports
with sufficiently smart URLs?

Rationale

I don’t have the power
to pull others onto a new platform.
But I can push useful data to others
in a way that inspires them to participate more directly with the platform

Proposal

Replace friendly Salesforce Reports and powerful NetSuite Saved Searches with a unified interface for viewing, editing, sharing, and managing:

  • versioned reports
  • personalized alerts
  • variant analyses

that are delivered via self-contained emails that also onboard people into greater use of the platform

Definitions

Friendly

  • Browseable
  • Drag and Drop
  • Live previews

Powerful

  • Complex formulas
  • Scaleable notifications
  • Easy joins and relabeling

Motivation

The main value of Quilt to my business
is as a point of leverage
to shift the culture of communication
from “zombie data” in tables
to “smart reports” in a repository

The Coherency Manifesto: Towards Communal Data Platforms

August 21, 2021 § Leave a comment

Version 1.0: Sep 11, 2021 (Interdependence Day)

As a community
who produces, consumes, and manages data
we hold these truths to be self-evident:

« Read the rest of this entry »

SyncHouse: MVC for Enterprise SaaS

May 2, 2021 § Leave a comment

A concrete proposal for Imagining a Data Resort as enforcing a Model-View-Controller architecture across multiple Software-as-a-Service applications. The key is replacing transient enterprise data integrations with a persistent “sync house,” and making that the one full-service Source of Truth for data, schemas, and business logic.

  1. Ingest data from Salesforce, NetSuite, etc. (e.g., Stitch/Talend, FiveTran)
  2. Store raw data in a LakeHouse (e.g., Databricks, Delta Lake; or just Redshift)
    1. Aka “ELT vs ETL
  3. Manage schemas via dbt (e.g., dbt Cloud)
  4. View and report on appropriate data (e.g., Mode, Data Studio)
  5. Push updates (reverse ETL) back to source applications (e.g., Celigo, Get Census)
« Read the rest of this entry »

D.R.I.P.: Platform Thinking for Lean Startups

May 21, 2015 § Leave a comment

Define

Terms, flows, roles, success criteria, open questions

Research

Characterize and identify Customer Zero

Implement

Enable Customer Zero to deploy a minimally viable app

Productize

Deliver a coherent and sustainable ecosystem

Transforming the Bay with Christ: A Platform Analysis

February 24, 2015 § 5 Comments

My series of four articles (plus a postscript) analyzing the regional spiritual renewal initiative Transforming the Bay with Christ:

  1. A Platform for Regional Transformation?
  2. From Platforms to Governance
  3. Trading Control for Authority
  4. The Process for Products
  5. TBC Postscript: A Missional Creed

TBC Postscript: A Missional Creed

February 24, 2015 § 2 Comments

I close out my series with a personal proposal for the “sacred space” around which to organize Transforming the Bay with Christ (TBC). 

I believe the mission of Transforming the Bay with Christ is to:

  • Mobilize, Unify, and Grow the Body of Christ
  • To Bless the San Francisco Bay Area
  • Through the Transforming Power of the Cross

« Read the rest of this entry »

TBC 4: The Process for Products

February 20, 2015 § 3 Comments

In this series I have been building a case that Transforming the Bay with Christ (TBC) should consider reframing itself as a startup building a platform for governance. In this, our final installment, I will discuss the process necessary to build such a product.

One of the key insights about entrepreneurship in the last decade is that a startup is not just a small version of a established business. Rather, a startup is an organization formed to search for a business model, rather than execute one.

In particular, this implies that startups should be designed to maximize learning by exploiting surprises. This is the exact opposite of a traditional business, which attempts to increase predictability by avoiding surprises.

To get the optimal structure, we need to be clear on:

  1. Which things we need to learn (the problem)
  2. How we are going to learn them (the process)
  3. Who will own the learning (the people)
  4. What will prove we have learned the right lessons (the product)

« Read the rest of this entry »

TBC 3: Trading Control for Authority

February 16, 2015 § 4 Comments

In this series we have been exploring the hypothesis that Transforming the Bay with Christ (TBC) would be most effective if structured as a platform designed for effective governance. Please note that these essays are purely an intellectual exercise on my part; I have no formal connection with or deep knowledge of TBC.

When designing systems of governance, the most important question is who holds which kinds of power. That is part of the genius of the American system of democracy: for all its flaws, the division of powers between the executive, legislative and judicial has produced a remarkably resilient (if horribly inefficient) system. In our last post, we focused on who holds the power. Now we will focus on which power is held. « Read the rest of this entry »

TBC 2: From Platforms to Governance

February 6, 2015 § 7 Comments

In our first installment of my series on Transforming the Bay with Christ (TBC), we talked about how platforms enable us to tackle problems and markets too big for any one entity to manage directly. Because of that, though, it is much harder to create a successful platform than it is to create a successful program. In this installment, we will talk about how to do that.

Characteristics of a Platform

The first thing to realize is that every platform is characterized by three distinct but interrelated factors:

  • Policy (governance)
  • Incentives (business)
  • Infrastructure (engineering)

The health of a platform is determined by how well these three factors support each other and the overall purpose of the platform.

This has two interesting implications:

  • Every complex human system (states, markets, corporations, etc.) can be considered a platform
  • The reason most platforms fail is that “wonks, suits, and geeks” only worry about their layer of the platform (politics, economics, or technology, respectively) and tend to despise or ignore the others

« Read the rest of this entry »

Where Am I?

You are currently browsing the Platforms category at iHack, therefore iBlog.