CTPP November 2013

ctpp@listserv.transportation.org

12 participants
21 discussions

Re: [CTPP] FW: Census Bureau News -- Census Bureau Media Advisory Commuting Products to be Embargoed

by Colson

The link is http://www.census.gov/newsroom/releases/archives/news_conferences/commuting… -----Original Message----- From: ctpp-news-bounces(a)chrispy.net [mailto:ctpp-news-bounces(a)chrispy.net] On Behalf Of Kevin Byrnes Sent: Tuesday, March 05, 2013 7:39 AM To: ctpp-news(a)chrispy.net Subject: Re: [CTPP] FW: Census Bureau News -- Census Bureau Media Advisory Commuting Products to be Embargoed Looking for download site for the co - to co JTW flow data released last night. Please provide link to site. Kevin Byrnes, AICP GWRC -----Original Message----- From: ctpp-news-bounces(a)chrispy.net [mailto:ctpp-news-bounces(a)chrispy.net] On Behalf Of Chuck Imbrogno Sent: Monday, March 04, 2013 4:58 PM To: ctpp-news(a)chrispy.net Subject: [CTPP] FW: Census Bureau News -- Census Bureau Media Advisory Commuting Products to be Embargoed Apologies to all. Meant to forward to folks in-house. Hit "send" before changing the "recipients" field. Chuck Imbrogno -----Original Message----- From: Chuck Imbrogno Sent: Monday, March 04, 2013 4:11 PM To: 'ctpp-news(a)chrispy.net' Subject: RE: [CTPP] Census Bureau News -- Census Bureau Media Advisory Commuting Products to be Embargoed Shannon, Chuck: FYI - See bottom of this e-mail chain for the "Media Advisory" from the Census Bureau regarding the Commuter Flow data that Tom Fontaine asked about earlier today. Data was "embargoed" by the Census Bureau. Available to the media at noon today, but not released publicly to everyone else (including us) until midnight tonight. Bob Schwartz should be downloading the file sometime in the morning. Chuck Imbrogno -----Original Message----- From: ctpp-news-bounces(a)chrispy.net [mailto:ctpp-news-bounces(a)chrispy.net] On Behalf Of Graham, Todd Sent: Friday, March 01, 2013 7:32 PM To: ctpp-news(a)chrispy.net Subject: Re: [CTPP] Census Bureau News -- Census Bureau Media Advisory Commuting Products to be Embargoed Census Bureau wil be releasing data and reports on commuting patterns next week. If you have Census PIO embargo access, you'll be able to dig into it as early as Monday afternoon. This is a new product -- so I'm not sure how the data will be structured -- anyone know? Enjoy. --Todd Graham Metropolitan Council Research ________________________________________ From: U.S. Census Bureau [census(a)subscriptions.census.gov] Sent: Friday, March 01, 2013 9:54 AM Subject: Census Bureau News -- **Census Bureau Media Advisory** Commuting Products to be Embargoed FOR IMMEDIATE RELEASE: FRIDAY, MARCH 1, 2013 Public Information Office CB13-39 301-763-3030 email: <pio(a)census.gov> ***CENSUS BUREAU MEDIA ADVISORY*** Commuting Products to be Embargoed The U.S. Census Bureau's Public Information Office will offer an embargo period next week for members of the media to view a series of commuting products. Statistics will be available for every county in the U.S. showing the number of workers that commute in or out and which counties those commuters travel to and from. Additional reports and tables, based on statistics from the American Community Survey, examine U.S. residents traveling across county and state lines to work. Specifically, the products present U.S. workers who have commutes of 60 minutes or longer and workers who have "mega commutes" of at least 90 minutes and 50 miles. Statistics will also be available for every county in the U.S. that show the number of workers that commute into or out of the county and which counties those commuters travel to and from. The reports and tables will be posted to the Census Bureau's embargo site at noon EST Monday, March 4. The public release will be at 12:01 a.m. EST Tuesday, March 5. Wire and distribution services are prohibited from distributing embargoed news releases and data files to subscribers before the public release date and time. If you are interested in scheduling a radio interview on Tuesday, March 5, please contact the U.S. Census Bureau Public Information Office at 301-763-3030. _______________________________________________ ctpp-news mailing list ctpp-news(a)ryoko.chrispy.net http://ryoko.chrispy.net/mailman/listinfo/ctpp-news _______________________________________________ ctpp-news mailing list ctpp-news(a)ryoko.chrispy.net http://ryoko.chrispy.net/mailman/listinfo/ctpp-news _______________________________________________ ctpp-news mailing list ctpp-news(a)ryoko.chrispy.net http://ryoko.chrispy.net/mailman/listinfo/ctpp-news

11 years, 2 months

[CTPP] Breaking News! SHP Files downloadable!

by Weinberger Penelope

Hi All, As many of you know, we have been having a problem with intermittent availability of SHP file downloads. This appears to be fixed. Penelope Z. Weinberger CTPP Program Manager AASHTO 202-624-3556 ctpp.transportation.org

11 years, 8 months

[CTPP] CTPP 2006-2010 FAQ is available on-line!

by Liang Long

Dear CTPP Users: Are you wondering what are the differences between CTPP 2006-2010 and Standard ACS products? Are you struggling Set A and Set B tables from the recent released CTPP? If so, please check our new FAQs on the FHWA website: http://www.fhwa.dot.gov/planning/census_issues/ctpp/faq/index.cfm As always, if you have any questions on the CTPP, feel free to contact me. Liang Long Cambridge Systematics, Inc. 4800 Hampden Lane Suite 800 Bethesda, MD 20814 tel 301 347 0100 fax 301 347 0101 FHWA 202-366-6971 e-mail llong(a)camsys.com www.camsys.com

11 years, 8 months

[CTPP] FW: Adventures in large datasets

by Weinberger Penelope

From: Howard Slavin [mailto:howard(a)caliper.com] Sent: Friday, November 15, 2013 11:32 AM To: 'ctpp-news(a)chrispy.net' Subject: RE: [CTPP] Adventures in large datasets Thanks to Krishnan for the kind mention of TransCAD. I wanted to let everyone know that Caliper is now distributing the ctpp flow data in matrix form for TransCAD users. While the data have been provided at the Transportation Analysis Zone level, only the Census tract to Census tract flow data are complete, and these are the data that Caliper is now providing to TransCAD users in the United States. The flow data are provided in the form of two TransCAD matrices that provide the estimated flows between all of the Census tracts in the U.S. and that distinguish estimated work trip flows with two different breakdowns by means of transportation and margins of error. The dimensions of these matrices are roughly 70,000 by 70,000, but remarkably they compress to only 1.5 GB or less and so are downloadable. With TransCAD, it is easy for users to subset the matrices to cover any geographic subarea of interest. These files can be downloaded by users at http://www.caliper.com/Press/census-ctpp-worker-flow-data-download.htm We expect to make the TAZ to TAZ flow data available as soon as it is corrected by the Census Bureau. Howard From: ctpp-news-bounces(a)chrispy.net<mailto:ctpp-news-bounces(a)chrispy.net> [mailto:ctpp-news-bounces(a)chrispy.net] On Behalf Of Krishnan Viswanathan Sent: Thursday, November 14, 2013 9:59 PM To: ctpp-news(a)chrispy.net<mailto:ctpp-news(a)chrispy.net> Subject: Re: [CTPP] Adventures in large datasets Mara Besides SQL server I have the following suggestions: 1) the ff package in R ( http://www.bnosac.be/index.php/blog/22-if-you-are-into-large-data-and-work-…) 2) HDF5 seems like a decent option though I have not used it. Link to rhdf5 ( http://bioconductor.org/packages/release/bioc/html/rhdf5.html). Also, SFCTA has some code for getting data into and out of HDF5 ( https://github.com/sfcta/TAutils/tree/master/hdf5) 3) I have found TransCAD to be efficient in processing large datasets. Hope this helps. Krishnan I downloaded the Maryland state raw data (the whole enchilada) that Penelope was good enough to provide me. It came with documentation that clearly explains what needs to be done but I am being hampered by the sheer size of the dataset. It's 10 GB and that's without going into joining tables, transposing them to meet my needs, etc. Even breaking the parts into different databases it can't be handled in Access. I can fit Part 1 into an ESRI geodatabase but I don't have the flexibility in linking tables that Access has. Does anyone have any suggestions for dealing with large databases? SQL server is one option. Are there others? Mara Kaminowitz, GISP GIS Coordinator ......................................................................... Baltimore Metropolitan Council Offices @ McHenry Row 1500 Whetstone Way Suite 300 Baltimore, MD 21230 410-732-0500 ext. 1030<tel:410-732-0500%20ext.%201030> mkaminowitz(a)baltometro.org<mailto:mkaminowitz(a)baltometro.org> www.baltometro.org <http://www.baltometro.org/> _______________________________________________ ctpp-news mailing list ctpp-news(a)ryoko.chrispy.net<mailto:ctpp-news(a)ryoko.chrispy.net> http://ryoko.chrispy.net/mailman/listinfo/ctpp-news

11 years, 8 months

Re: [CTPP] CTPP 2006-2010 Software Cheatsheets

by Tom Faella

Dear CTPP listservers: These are very helpful! Thank you Liang, Tom Faella Executive Director, La Crosse Area Planning Committee (LAPC) La Crosse County Administrative Center 400 4th Street North, Room 2300 La Crosse, WI 54601 (608) 785-5977 (608) 793-6525 (Fax) www.LAPC.org tfaella(a)lacrossecounty.org<mailto:tfaella(a)lacrossecounty.org> From: ctpp-news-bounces(a)chrispy.net [mailto:ctpp-news-bounces(a)chrispy.net] On Behalf Of Liang Long Sent: Wednesday, November 13, 2013 3:00 PM To: ctpp-news(a)chrispy.net Subject: [CTPP] CTPP 2006-2010 Software Cheatsheets Dear CTPP Users: Please find attached the CTPP 2006-2010 software cheat sheets that we have developed. For example, if you are trying to get tract to tract or TAZ to TAZ flows for multiple counties, you may find the cheat sheet for part 3 useful. Feel free to contact me if you have any questions. Liang Long Cambridge Systematics, Inc. 4800 Hampden Lane Suite 800 Bethesda, MD 20814 tel 301 347 0100 fax 301 347 0101 FHWA 202-366-6971 e-mail llong(a)camsys.com<mailto:llong(a)camsys.com> www.camsys.com PRIVATE AND CONFIDENTIAL This e-mail and attachments are intended for the addressed recipient only. If you are not the correct recipient please notify the sender of the delivery error and delete this message. Improper disclosure, copying, distribution, retransmission, or use of information from this e-mail is Prohibited, and may result in liability and damages for misuse of this information.

11 years, 8 months

[CTPP] Fwd: Call for abstracts for the ACS Data Users Conference

by Ed Christopher

This may be of interest to some. > > ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > ACS Data Users Conference > > May 29-30, 2014 > Holiday Inn Capitol, Washington D.C. > > Call for Abstracts! > Deadline: January 15 > Notification Date: February 15 > > To submit an abstract, visit www.acsdatausers.org. The $200 conference registration fee will be waived for persons presenting at the meeting. > > The American Community Survey (ACS) Data Users Conference will bring together ACS data users and staff from the U.S. Census Bureau to improve understanding of the value and utility of ACS data and to promote information sharing among data users about key ACS data issues and applications. The conference will include a mix of invited and contributed presentations by ACS data users and Census Bureau staff. > > We are inviting abstract submissions on the following topics (submissions on other applications of ACS data are also welcome): > Using the ACS to look at trends over time, including multiyear estimates > Issues and limitations in using small-area ACS data > How data users have dealt with changes in ACS survey questions > Aggregating ACS estimates and calculating margins of error > Using ACS in GIS and other mapping applications > How to access ACS from different sources > Applications of ACS data in different topical areas: Health insurance and disability; Income and poverty; employment and workforce development; and Migration patterns and trends > Please forward this announcement to others who may be interested. More information about the conference, including registration information, will be available in the coming weeks. > >

11 years, 8 months

Re: [CTPP] Adventures in large datasets

by Krishnan Viswanathan

Hary Thanks for the clarification and pointing out the drawbacks with using HDF5. I am all for open source software so if MySQL etc. works then great. I found TransCAD really fast in processing large (though not using 10 GB) datasets esp. when sorting and found it much faster than standard statistical packages. Krishnan On Fri, Nov 15, 2013 at 9:03 AM, hprawiranata mitcrpc.org < hprawiranata(a)mitcrpc.org> wrote: > Krishnan, > > HDF was data format, all climate data is stored on HDF format (I > experience writing C code for this data format on SPARC computers long > time ago and from HDF I converted to MySQL ! ). CTPP is just a flat > data set, txt. Converting into HDF is a different complicated story > and on a wrong direction. > > Linking data tables from many tables has to be done using database > engine. MS Access won't cut it. M$ SQL is expensive. Free, Open Source > is the only way and fast. > > TransCAD... I never test the limit. But linking simple tables is ok. > Large and many tables ? Better use data modeler software and link to > database engine. > > Hary(ono) Prawiranata > Transportation Analyst/Modeler > Tri-County Regional Planning Commission > 3135 Pine Tree Rd. Ste 2C > Lansing, MI 48911 > > On Thu, Nov 14, 2013 at 9:58 PM, Krishnan Viswanathan > <krisviswanathan(a)gmail.com> wrote: > > Mara > > > > Besides SQL server I have the following suggestions: > > 1) the ff package in R ( > > > http://www.bnosac.be/index.php/blog/22-if-you-are-into-large-data-and-work-… > ) > > 2) HDF5 seems like a decent option though I have not used it. Link to > rhdf5 > > ( http://bioconductor.org/packages/release/bioc/html/rhdf5.html). Also, > > SFCTA has some code for getting data into and out of HDF5 ( > > https://github.com/sfcta/TAutils/tree/master/hdf5) > > 3) I have found TransCAD to be efficient in processing large datasets. > > > > Hope this helps. > > > > Krishnan > > > > I downloaded the Maryland state raw data (the whole enchilada) that > Penelope > > was good enough to provide me. It came with documentation that clearly > > explains what needs to be done but I am being hampered by the sheer size > of > > the dataset. It's 10 GB and that's without going into joining tables, > > transposing them to meet my needs, etc. Even breaking the parts into > > different databases it can't be handled in Access. I can fit Part 1 > into an > > ESRI geodatabase but I don't have the flexibility in linking tables that > > Access has. > > > > > > > > Does anyone have any suggestions for dealing with large databases? SQL > > server is one option. Are there others? > > > > > > > > Mara Kaminowitz, GISP > > GIS Coordinator > > ......................................................................... > > Baltimore Metropolitan Council > > Offices @ McHenry Row > > 1500 Whetstone Way > > Suite 300 > > Baltimore, MD 21230 > > 410-732-0500 ext. 1030 > > mkaminowitz(a)baltometro.org > > www.baltometro.org > > > > > > > > > > _______________________________________________ > > ctpp-news mailing list > > ctpp-news(a)ryoko.chrispy.net > > http://ryoko.chrispy.net/mailman/listinfo/ctpp-news > > > > > > _______________________________________________ > > ctpp-news mailing list > > ctpp-news(a)ryoko.chrispy.net > > http://ryoko.chrispy.net/mailman/listinfo/ctpp-news > > > _______________________________________________ > ctpp-news mailing list > ctpp-news(a)ryoko.chrispy.net > http://ryoko.chrispy.net/mailman/listinfo/ctpp-news > -- Krishnan Viswanathan 5628 Burnside Circle Tallahassee FL 32312

11 years, 8 months

[CTPP] FW: Call for Abstracts: ACS Data Users Conference

by Weinberger Penelope

fyi From: ACS Data Users Group [mailto:ACS_Data_Users_Group(a)mail.vresp.com] Sent: Friday, November 15, 2013 3:01 PM To: Weinberger Penelope Subject: Call for Abstracts: ACS Data Users Conference [ACS data users logo] ACS Data Users Conference May 29-30, 2014 Holiday Inn Capitol, Washington D.C. Call for Abstracts! Deadline: January 15 Notification Date: February 15 To submit an abstract, visit www.acsdatausers.org.<http://cts.vresp.com/c/?PopulationReferenceB/d13a469f3e/2af176bb3e/c961d350…> The $200 conference registration fee will be waived for persons presenting at the meeting. The American Community Survey (ACS) Data Users Conference will bring together ACS data users and staff from the U.S. Census Bureau to improve understanding of the value and utility of ACS data and to promote information sharing among data users about key ACS data issues and applications. The conference will include a mix of invited and contributed presentations by ACS data users and Census Bureau staff. We are inviting abstract submissions on the following topics (submissions on other applications of ACS data are also welcome): * Using the ACS to look at trends over time, including multiyear estimates * Issues and limitations in using small-area ACS data * How data users have dealt with changes in ACS survey questions * Aggregating ACS estimates and calculating margins of error * Using ACS in GIS and other mapping applications * How to access ACS from different sources * Applications of ACS data in different topical areas: Health insurance and disability; Income and poverty; employment and workforce development; and Migration patterns and trends Please forward this announcement to others who may be interested. More information about the conference, including registration information, will be available in the coming weeks. ________________________________ Click to view this email in a browser<http://hosted.verticalresponse.com/1056151/d13a469f3e/521474375/2af176bb3e/> If you no longer wish to receive these emails, please reply to this message with "Unsubscribe" in the subject line or simply click on the following link: Unsubscribe<http://cts.vresp.com/u?d13a469f3e/2af176bb3e/mlpftw> Click here<http://oi.vresp.com/f2af/v4/send_to_friend.html?ch=d13a469f3e&lid=521474375…> to forward this email to a friend Population Reference Bureau 1875 Connecticut Ave., NW Suite 520 Washington, District of Columbia 20009 USA Read<http://www.verticalresponse.com/content/pm_policy.html> the VerticalResponse marketing policy. [Try Email Marketing with VerticalResponse!]<http://www.verticalresponse.com/landing/ef/?mm/d13a469f3e&utm_campaign=foot…> [http://cts.vresp.com/o.gif?d13a469f3e/2af176bb3e/mlpftw]

11 years, 8 months

Re: [CTPP] Adventures in large datasets

by hprawiranata mitcrpc.org

Actually, I have downloaded whole Michigan CTPP flat file ! Now, in extracting process. processing it, total data 15,523 MB !, I will dump it into MySQL and play with data modeler tools I have. Thanks ! Hary(ono) Prawiranata Transportation Analyst/Modeler Tri-County Regional Planning Commission 3135 Pine Tree Rd. Ste 2C Lansing, MI 48911 On Fri, Nov 15, 2013 at 8:52 AM, hprawiranata mitcrpc.org <hprawiranata(a)mitcrpc.org> wrote: > My experience handling large data set (climate data set, on linux, > with parallel computers, and with R, Python) was using free software > MySQL (or PostgreSQL, mostly I use MySQL), for easy access and > management use MySQL workbench ( > http://www.mysql.com/products/workbench/ ). > If you already install MOVES 2010, you can use MOVES MySQL engine to > dump your data. I use this MySQL to process vehicle registration data > (~9 million records of Michigan vehicles), it is faster than MS Access > cannot handle it.And it is on Windows.. (I have another version on > Linux, too). > > I'm interested to dump Michigan data too, how big is this ? Penelope ? > > Hary(ono) Prawiranata > Transportation Analyst/Modeler > Tri-County Regional Planning Commission > 3135 Pine Tree Rd. Ste 2C > Lansing, MI 48911 > > > On Thu, Nov 14, 2013 at 2:19 PM, Mara Kaminowitz > <mkaminowitz(a)baltometro.org> wrote: >> I downloaded the Maryland state raw data (the whole enchilada) that Penelope >> was good enough to provide me. It came with documentation that clearly >> explains what needs to be done but I am being hampered by the sheer size of >> the dataset. It's 10 GB and that's without going into joining tables, >> transposing them to meet my needs, etc. Even breaking the parts into >> different databases it can't be handled in Access. I can fit Part 1 into an >> ESRI geodatabase but I don't have the flexibility in linking tables that >> Access has. >> >> >> >> Does anyone have any suggestions for dealing with large databases? SQL >> server is one option. Are there others? >> >> >> >> Mara Kaminowitz, GISP >> GIS Coordinator >> ......................................................................... >> Baltimore Metropolitan Council >> Offices @ McHenry Row >> 1500 Whetstone Way >> Suite 300 >> Baltimore, MD 21230 >> 410-732-0500 ext. 1030 >> mkaminowitz(a)baltometro.org >> www.baltometro.org >> >> >> >> >> _______________________________________________ >> ctpp-news mailing list >> ctpp-news(a)ryoko.chrispy.net >> http://ryoko.chrispy.net/mailman/listinfo/ctpp-news >>

11 years, 8 months

Re: [CTPP] Adventures in large datasets

by hprawiranata mitcrpc.org

Krishnan, HDF was data format, all climate data is stored on HDF format (I experience writing C code for this data format on SPARC computers long time ago and from HDF I converted to MySQL ! ). CTPP is just a flat data set, txt. Converting into HDF is a different complicated story and on a wrong direction. Linking data tables from many tables has to be done using database engine. MS Access won't cut it. M$ SQL is expensive. Free, Open Source is the only way and fast. TransCAD... I never test the limit. But linking simple tables is ok. Large and many tables ? Better use data modeler software and link to database engine. Hary(ono) Prawiranata Transportation Analyst/Modeler Tri-County Regional Planning Commission 3135 Pine Tree Rd. Ste 2C Lansing, MI 48911 On Thu, Nov 14, 2013 at 9:58 PM, Krishnan Viswanathan <krisviswanathan(a)gmail.com> wrote: > Mara > > Besides SQL server I have the following suggestions: > 1) the ff package in R ( > http://www.bnosac.be/index.php/blog/22-if-you-are-into-large-data-and-work-…) > 2) HDF5 seems like a decent option though I have not used it. Link to rhdf5 > ( http://bioconductor.org/packages/release/bioc/html/rhdf5.html). Also, > SFCTA has some code for getting data into and out of HDF5 ( > https://github.com/sfcta/TAutils/tree/master/hdf5) > 3) I have found TransCAD to be efficient in processing large datasets. > > Hope this helps. > > Krishnan > > I downloaded the Maryland state raw data (the whole enchilada) that Penelope > was good enough to provide me. It came with documentation that clearly > explains what needs to be done but I am being hampered by the sheer size of > the dataset. It's 10 GB and that's without going into joining tables, > transposing them to meet my needs, etc. Even breaking the parts into > different databases it can't be handled in Access. I can fit Part 1 into an > ESRI geodatabase but I don't have the flexibility in linking tables that > Access has. > > > > Does anyone have any suggestions for dealing with large databases? SQL > server is one option. Are there others? > > > > Mara Kaminowitz, GISP > GIS Coordinator > ......................................................................... > Baltimore Metropolitan Council > Offices @ McHenry Row > 1500 Whetstone Way > Suite 300 > Baltimore, MD 21230 > 410-732-0500 ext. 1030 > mkaminowitz(a)baltometro.org > www.baltometro.org > > > > > _______________________________________________ > ctpp-news mailing list > ctpp-news(a)ryoko.chrispy.net > http://ryoko.chrispy.net/mailman/listinfo/ctpp-news > > > _______________________________________________ > ctpp-news mailing list > ctpp-news(a)ryoko.chrispy.net > http://ryoko.chrispy.net/mailman/listinfo/ctpp-news >

11 years, 8 months

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

CTPP November 2013