From “Zombie Data” to “Smart Reports”
October 4, 2021 § Leave a comment
The bane of my IT existence is a business user who says, “Please get me the latest version of <random Excel file I have never seen before, named using idiosyncratic or ambiguous words>. Oh, and I need it tomorrow or else we won’t {make our numbers | pass our audit | satisfy the board}.”
I call this “zombie data” because it:
- Lacks any self-awareness
- Doesn’t remember where it came from
- Has no relationship to its current context
- Infects everyone it touches with that same mindlessness.
Being Human: A Curriculum
September 21, 2021 § Leave a comment
Reframing spirituality as the culturally-neutral, teachable practice of becoming more human, as expressed in:
- 3 Capabilities
- 4 Attributes per Capability
- Prerequisite
- Task
- Technique
- Mindset
- 12 Learning Outcomes
Inspired by the Minerva Baccalaureate, especially their focus on Content over Context.
« Read the rest of this entry »Religious Entrepreneurship in Nevil Shute’s Round the Bend
September 8, 2021 § Leave a comment
I am currently podcasting a book club with my father on Nevil Shute‘s novel Round the Bend. The framing question we are using for our discussions is “What is this book about?” There are a number of different themes that have come up (e.g., technology, love, and racism), but my current hypothesis is that the book is primarily about “Religious Entrepreneurship”, a sort of cousin to the Spiritual Entrepreneurship I studied with Hatchery LA.
Here (four chapters in) is how and why I think the author wrote this book. This is all purely hypothetical on my part; I have not done any actual research on his methods. However, this reflects the kind of process that I could see myself going through, based on comments the author made about the context in which the protagonist built his business.
« Read the rest of this entry »The Reporting Control Center
September 5, 2021 § 1 Comment
aka Quilt Data Hub or Lightdash 2.0?
Challenge
Can I evangelize
a corporate data platform
by just emailing out reports
with sufficiently smart URLs?
Rationale
I don’t have the power
to pull others onto a new platform.
But I can push useful data to others
in a way that inspires them to participate more directly with the platform
Proposal
Replace friendly Salesforce Reports and powerful NetSuite Saved Searches with a unified interface for viewing, editing, sharing, and managing:
- versioned reports
- personalized alerts
- variant analyses
that are delivered via self-contained emails that also onboard people into greater use of the platform
Definitions
Friendly
- Browseable
- Drag and Drop
- Live previews
Powerful
- Complex formulas
- Scaleable notifications
- Easy joins and relabeling
Motivation
The main value of Quilt to my business
is as a point of leverage
to shift the culture of communication
from “zombie data” in tables
to “smart reports” in a repository
The Coherency Manifesto: Towards Communal Data Platforms
August 21, 2021 § Leave a comment
Version 1.0: Sep 11, 2021 (Interdependence Day)
As a community
who produces, consumes, and manages data
we hold these truths to be self-evident:
Psycho-Analytic Engineering (Coalesce 2021)
June 6, 2021 § Leave a comment
Using Data to Differentiate Our Selves
Keynote Talk Proposal for Coalesce 2021
Based on “DBT as Organizational Therapy“
« Read the rest of this entry »DBT as the “Couch” for Organizational Therapy
May 13, 2021 § 1 Comment
Or, “How ELTT is the Key to World Peace”
Draft Submission Script for Coalesce 2021 « Read the rest of this entry »
Configuring DataBricks on AWS
May 5, 2021 § Leave a comment
Despite the excellent QuickStart tools, this was way harder than I thought. For some reason I had the worst difficulty creating a Workspace on AWS for Databricks.
Here are some tips that might help others who get stuck.
« Read the rest of this entry »SyncHouse: MVC for Enterprise SaaS
May 2, 2021 § Leave a comment
A concrete proposal for Imagining a Data Resort as enforcing a Model-View-Controller architecture across multiple Software-as-a-Service applications. The key is replacing transient enterprise data integrations with a persistent “sync house,” and making that the one full-service Source of Truth for data, schemas, and business logic.
- Ingest data from Salesforce, NetSuite, etc. (e.g.,
Stitch/Talend, FiveTran) - Store raw data in a LakeHouse (e.g., Databricks, Delta Lake; or just Redshift)
- Aka “ELT vs ETL“
- Manage schemas via dbt (e.g., dbt Cloud)
- View and report on appropriate data (e.g., Mode, Data Studio)
- Push updates (reverse ETL) back to source applications (e.g.,
Celigo, Get Census)
Imagining a Data Resort
April 21, 2021 § 1 Comment
A data resort is where data comes to get pampered, so that it is prepared to get back to work.
Motivation
The good news is that I finally understand how we really need to be managing all the business data in my organization. The bad news is that I don’t know how to articulate that in terms of industry-standard terminology (examples below). Worse, I’d probably use the wrong term (or the right term incorrectly) leading to endless rounds of frustrating conversation.
Therefore I’ve coined a new term (“data resort“) which I can define as the exact thing I want. Hopefully you my dear readers can help translate that into something concrete I can efficiently buy+build today!
Let me know your suggestions in the comments or via email.
« Read the rest of this entry »
You must be logged in to post a comment.