THIS IS AN ARCHIVED SITE
This site contains information from January 2009-December 2014. Click HERE to go the CURRENT commerce.gov website.

Blog Category: Big data

Advocating the Transformative Power of Commerce Data at NYC STRATA + Hadoop World Conference

Submitted on October 15, 2014 - 3:45pm
Categories:
Jennifer van der Meer and Under Secretary Mark Doms at NYC STRATA + Hadoop World Conference

Cross posted from ESA.gov

Under Secretary Mark Doms participated in a high level data discussion this morning at the Strata+Hadoop World Conference in New York City. Before an audience of 500 leading technologists and data programmers, Under Secretary Doms talked with host Jennifer van der Meer, Adjunct Professor at NYU's Interactive Telecommunications Program and CEO of Reason Street, to explore the Department of Commerce's strategic data plan and Doms' efforts to move the federal statistical system into the era of big data.
 
Doms noted that the US Department of Commerce has long been a powerhouse for government data, trailblazing the use of government statistics and analysis to help everyone make more informed decisions. Now, in the era of big data, with large volumes of data collected and analyzed by the private sector, by citizens themselves, the agency, with Doms leadership, is working to position itself as a leader in the federal data space. Jennifer van der Meer asked the Under Secretary about Commerce's plans to hire its first Chief Data Officer, stand up a Data Advisory Council populated with private sector and academic data leaders, and ways the Department is looking to team with the private sector to better collect, disseminate, and analyze Commerce data.
 
Doms went on to highlight the fact that challenges facing companies and our society often do not fit neatly in the "buckets" represented by the various federal agencies. Commerce has data that, say, could be meshed with Department of Education data, to tackle our nation's skills gap or help students determine which majors have the best return on investment. Doms noted Commerce's involvement with the White House's Office of Science and Technology Policy and their efforts to coordinate interagency discussion to share best practices and tackle cross-agency challenges. Doms pointed out this coordination is critical to unleashing the positive benefits of federal data, with the next step being to figure out how to incorporate private datasets and get greater corporate buy-in to the open data movement currently underway at the federal, state and local level.

Under Secretary Doms closed out the discussion by making the case that the federal government must remain a leader in data. Like our basic scientific research, the building and maintaining of our nation's highways and water treatment facilities, and rural postal delivery, providing comprehensive data on our people, economy and the planet will continue to be a core federal mission. This information is critical to decision making by every business, government, and citizen, and the private sector simply does not have the financial incentive to fill this role. Doms thanked Jennifer and the audience for a lively discussion, one that further informed his efforts, under Secretary Pritzker's leadership, to revolutionize data at the Department of Commerce.

  • Share

Data Innovation Summit: Taking Advantage of Boston’s Big Data Movement

Submitted on September 25, 2014 - 5:25pm
Categories:
Under Secretary Mark Doms Speaks at the Big Data Innovation Summit in Boston, MA

Guest blog post by Mark Doms, Under Secretary for Economic Affairs

Cross-posted from ESA.gov.

I was honored to deliver the keynote address this morning to over 800 of today’s data leaders.  In preparation I had one goal: to tell the story of data -- how far the Department of Commerce has come and it’s potential for the future. These are exciting times. Over the past decade, jobs in data fields have grown at a rate 6 times faster than the economy as a whole, and these jobs pay 73% more than the typical American job.

The federal government provides fundamental statistical building blocks about our population, our economy, and our climate. This information is so pervasive that people often are unaware they are using government data. For instance, one survey found 301 billion weather forecasts are consumed per year -- information that is delivered by an array of sources, but begins with the National Weather Service, part of the Commerce Department’s National Oceanic and Atmospheric Administration. Indeed, many of the companies attending the Big Data Innovation Summit today in Boston rely heavily upon government data.  

Since 1790, when the first Census occurred, our government has taken the time to collect information that tells the demographic and economic story of our nation. We take this commitment very seriously at the Department of Commerce and we welcome user input.  

read more
  • Share

New Commerce Department report explores huge benefits, low cost of government data

Submitted on July 14, 2014 - 5:59pm
Categories:
Fostering Innovation, Creating Jobs, Driving Better Decisions: The Value of Government Data

Cross post by Mark Doms, Under Secretary for Economic Affairs

Today we are pleased to roll out an important new Commerce Department report on government data. “Fostering Innovation, Creating Jobs, Driving Better Decisions: The Value of Government Data,” arrives as our society increasingly focuses on how the intelligent use of data can make our businesses more competitive, our governments smarter, and our citizens better informed. 

And when it comes to data, as the Under Secretary for Economic Affairs, I have a special appreciation for the Commerce Department’s two preeminent statistical agencies, the Census Bureau and the Bureau of Economic Analysis. These agencies inform us on how our $17 trillion economy is evolving and how our population (318 million and counting) is changing, data critical to our country. Although “Big Data” is all the rage these days, the government has been in this  business for a long time: the first Decennial Census was in 1790, gathering information on close to four million people, a huge dataset for its day, and not too shabby by today’s standards as well. 

Just how valuable is the data we provide? Our report seeks to answer this question by exploring the range of federal statistics and how they are applied in decision-making. Examples of our data include gross domestic product, employment, consumer prices, corporate profits, retail sales, agricultural supply and demand, population, international trade and much more.

read more
  • Share

Response to NOAA's data RFI - "let's get started." And we are!

Submitted on June 23, 2014 - 2:45pm
Categories:
National Weather Service Data Visualization

Did you know that NOAA gathers 20 terabytes of data every day - twice the data of the entire printed collection of the United States Library of Congress? This environmental intelligence comes from a wide variety of sources including: Doppler radar systems, weather satellites, buoy networks and stations, computer models, tide gauges, real-time weather stations, as well as ships and aircraft. This network provides valuable and critical data that are instrumental in protecting lives and property across the country. But only a small percentage is easily accessible to the public and, as demand increases for this data and information, NOAA recognized it needed to find ways to effectively and efficiently distribute this data to decision makers and industries.

With that in mind, this past February NOAA announced a new effort to unleash the power of its data to foster innovation, create new industries and job opportunities. NOAA issued a Request for Information, or an RFI, to engage private industry to help make NOAA's data available in a rapid, scalable manner to the public. Through this process, American companies were asked to provide possible solutions for NOAA to be able to turn this untapped information into usable products or services.

So what does this mean to the economy? According to a 2013 McKinsey Global Institute Report, open data could add more than $3 trillion in total value annually to the education, transportation, consumer products, electricity, oil and gas, healthcare, and consumer finance sectors worldwide. If more of this data could be efficiently released, organizations will be able to develop new and innovative products and services to help us better understand our planet and keep communities resilient from extreme events.

NOAA received more than 70 responses to the RFI that closed on March 31st.  Responses came from industry and academia and ranged from single organizations to broad, integrated teams. NOAA has reviewed the responses and is continuing to engage with industry to elicit feedback for the best way to make this data accessible and useful.

Overall, respondents to the RFI provided a clear message - get started. And we heard this again at the Open Data Roundtable last week at the White House. So NOAA is talking to other agencies and formulating a plan for implementing a new, innovative model of public-private partnership around open government data, all in support of the Obama administration’s efforts to make data more accessible. NOAA intends to incrementally implement this partnership by enabling the government and industry to work together by testing the best methods for not just making data available, but creating an ecosystem around the data that will make a meaningful and lasting impact on the economy.

Big Data is Big Business for Commerce

Submitted on April 8, 2014 - 6:15pm
Categories:
Under Secretary for Economic Affairs Mark Doms (center) along with Erie Meyer, Joel Gurin, Waldo Jaquith, and Daniel Castro at the Center for Data Innovation hosted “The Economic Benefits of Open Data” event

Guest blog post by Mark Doms, Under Secretary for Economic Affairs

Big Data and Open Data are all the rage these days. However, Commerce was into Big Data before Big Data was cool. As far back as 1790, we began collecting data on patents in the U.S. and the Census Bureau conducted the first Decennial Census the same year. In 1870, the National Weather Service was created – which today is one of the biggest data producing agencies around.

Back then, our economy was based largely on agriculture. Over the years, our economy evolved through the industrial revolution, later giving rise to the strong service sector. Today, we are at the nascent stages of the next era in our economic growth, the information age. On a daily basis, there is an ever-increasing amount of data becoming available, and the demand for data is increasing exponentially. We have before us both great opportunity and fascinating challenges to understand how best to harness this national resource. This is a key focus of Commerce’s Open for Business Agenda.

You may not know it, but the Department of Commerce is home to many agencies that are your primary source for data that you likely use every day.

For example:

  • How many people live in the U.S. or in your hometown? You might know the Census Bureau is the authority on population, but did you know the Census Bureau’s data goes well beyond just population? Census also produces huge volumes of data on our economy, demographics, and fascinatingly insightful data describing our communities – or, if you are a business, your customers.
  • The Bureau of Economic Analysis is a little know agency that produces key economic data and many of the closely watched economic indicators that move markets, drive investment decisions and guide economic policy. Do you know which industries are the leading sources of income in your community, or to your customers? BEA data can tell you.
  • The National Oceanic and Atmospheric Administration, or NOAA, is your primary source for weather, ocean and climate data – they are collecting data every minute of every day from land, sea, and even spaced-based sensors. When you hear the local forecast or hear about severe weather warning, that is NOAA data informing you about your environment in real time.
  • The National Institute of Standards and Technology, locally known as NIST, is our nation’s authority on broad swaths of scientific, cyber, and physical data – including, officially, what time it is.
  • We also have data on patents going back more than 200 years at the U.S. Patent and Trademark Office, which is a gold mine of inspiration for innovation.
  • Other agencies in Commerce provide data on economic development, minority businesses, trade, and telecommunications and the Internet.

On any given day, the Department will generate in excess of 20 terabytes of data, and sometimes much more. Yet, we think we can do more with this resource. We want to take every step we can to open access to it to the entrepreneurs and innovators of America, as we are pretty convinced that there is huge unmet value and potential. We understand that a huge part of the value of data is when it is not seen alone, but as part of a rich tapestry of information. We believe that there is great opportunity to solve problems, innovate new businesses, and improve data-driven decision-making, and we are committed to that path.

That is why I was so glad to be a part of today’s launch of the Open Data 500 Project, housed out of the GovLab at NYU. This exciting project has verified what we were certain must be true: That hundreds of American companies are using Commerce data every day to innovate and deliver important goods and services to their customers.

read more
  • Share

The Commerce Department’s Strategic Plan: The Value of Government Data

Submitted on March 24, 2014 - 1:59pm
Categories:
The Average Daily Cost, Per Person, of the Principal Statistical Agencies is Three Cents

Guest blog post by Mark Doms, Under Secretary for Economic Affairs

Last week, Commerce Secretary Penny Pritzker unveiled the Department’s America is Open for Business:  Strategic Plan, Fiscal Years 2014-2018.  One of the plan’s five priority areas is a redefinition of how we manage, optimize and enable public access to our treasure trove of data.  The Commerce Department is fortunate to have numerous agencies that provide data that are critical to the information economy, such as:

  • The U.S. Census Bureau and Bureau of Economic Analysis (BEA) demographic and economic statistics;
  • National Oceanic and Atmospheric Administration (NOAA) weather, ocean and climate information; 
  • National Institute of Standards and Technology (NIST) scientific data;
  • National Technical Information Service (NTIS) information; and
  • U.S. Patent and Trademark Office (USPTO) patent databases. 

Specifically, the plan pledges to “improve government, business, and community decisions and knowledge by transforming Department data capabilities and supporting a data-enabled economy.”  Success has three dimensions.  First, everyone in our country should have easy access to reliable information about their communities, about their climate, and about how these are changing.  Second, every business should have easy access to reliable information on their market, potential markets, scientific information, and changing economic conditions.  Further, new data-based businesses should be able to easily pull our data, combine it with other information, and make new products to compete in the private marketplace.  Third, and finally, every government should have easy access to the information they need to better serve their communities and to assess the efficacy of their programs.  More simply put, success is making our data accessible in ways that make our businesses more competitive, our governments smarter, and our citizens more informed.

How will that be achieved?  The first component is to transform DOC’s data capacity to make our data more accessible and usable.  The second component of the data strategic plan is for us to use data to make government smarter.  The third objective of our plan is to develop better collaboration and feedback loops with the private sector; to create timely, relevant, and accessible products and services.  Many specific initiatives are well underway.  For example, NOAA already is seeking private-sector input on new public-private partnership models to make more weather and climate data available.  NIST is spearheading the development of Big Data standards. <--break->

read more
  • Share

Secretary Pritzker Outlines Bold Policy Agenda Focused on U.S. Trade and Investment, Innovation, and Data

Submitted on November 14, 2013 - 3:30pm
Categories:
Secretary Pritzker Outlines Bold Policy Agenda Focused on U.S. Trade and Investment, Innovation, and Data

Secretary of Commerce Penny Pritzker today outlined a bold new policy agenda for the Department of Commerce, centered on U.S. trade and investment, innovation, and data.  This â€śOpen for Business” Agenda reflects the department’s role as the voice of business, and the Obama Administration’s focus on economic growth and job creation. Additionally, this new vision recognizes the demands of a globally competitive economy.

Among the new initiatives Secretary Pritzker announced are a revitalized National Export Initiative, an enhanced and expanded program to attract foreign investment, a first-of-its-kind, Commerce effort to ensure skills training programs meet industry needs, and a focus on public-private partnerships that enable businesses and communities to make better use of government data.

Secretary Pritzker also committed to leading a robust environmental agenda at Commerce, and to ensuring that operational excellence and public accountability are top priorities of departmental leadership. 

read more

Census Bureau Releases Its First Mobile App Providing Real-Time Statistics on U.S. Economy

Submitted on August 9, 2012 - 1:32pm
Categories:
Logo: America's Economy

The Department of Commerce's U.S. Census Bureau today released its first-ever mobile application, "America's Economy," which will provide constantly updated statistics on the U.S. economy, including monthly economic indicators, trends, along with a schedule of upcoming announcements. The app, which is currently available for Android mobile device users, combines statistics from the U.S. Department of Commerce's Census Bureau, Bureau of Economic Analysis, and the U.S. Department of Labor's Bureau of Labor Statistics.

America's Economy is the first mobile app from the Census Bureau that provides smartphone and tablet users with the real-time government statistics that drive business hiring, sales and production decisions and assist economists, researchers, planners and policymakers. The economic indicators track monthly and quarterly trends in industries, such as employment, housing construction, international trade, personal income, retail sales and manufacturing.

The America's Economy app has been developed as part of the Census Bureau's Web Transformation Project and fulfills a key goal of President Obama's recently announced Digital Strategy to provide federal employees and the general public with greater access to government information and services. The creation of this app is also consistent with the Census Bureau's longtime mission of providing accurate statistics about the nation's growth and changes using 21st century technology to make that information available more quickly and easily. Read the full press release. America's Economy is available now for Android users and is expected be available for Apple smartphone and tablet users in the Apple App Store in the coming weeks.