The Diocese of Brooklyn was established in 1853 and originally included Kings, Queens, Nassau and Suffolk counties.  The Most Rev. John Loughlin was the first Diocesan Bishop and laid the foundation for what became the largest fully urban diocese in the country in 1957 when Nassau and Suffolk counties became the Diocese of Rockville Centre shortly after the death of Archbishop Thomas E. Molloy, the third Bishop of the Diocese. The need to ensure a proper education system for the training of future priests led the second Bishop of Brooklyn, the Most Reverend Charles E. McDonnell, to establish Cathedral College of the Immaculate Conception in 1914 at a new building located at 555 Washington Avenue in Brooklyn. Auxiliary Bishop (later Cardinal) George W. Mundelein was the first rector. Modeled after the traditional European Minor Seminary system, the College included the four years of high school and the first two years of college. In 1963, with an eye to developing a seminary system more in keeping with the United States system of education, he opened a four-year high school at Cathedral Preparatory Seminary in Elmhurst. This was a full four-year high school seminary and brother school to Brooklyn Cathedral.

In 1967 Archbishop McEntegart established Cathedral College of the Immaculate  Conception in Douglaston. This full four-year college seminary was to serve the Dioceses of Brooklyn and Rockville Centre as well as the Archdiocese of New York. At its  peak in the early 1970’s the college housed more than 370 students and offered baccalaureate degrees in nearly a dozen majors. The decline in the number of those answering the call to the priesthood led to the closing of Brooklyn Cathedral in 1985 and Cathedral College in 1987, with the last degrees granted in 1988.

In 1987, Bishop Francis J. Mugavero, the fifth Bishop of Brooklyn, established the Immaculate Conception Center at the site of the former Cathedral College. A portion of the property was sold to fund the renovation of the building and the Diocese of Brooklyn had its first pastoral center. 

In addition to the Bishops Mugavero/Mulrooney Residences for priests and Cathedral Seminary Residence of the Immaculate Conception, the Center serves as a retreat and conference center. It is home to the Diocesan Tribunal, Pastoral Institute, Institutional Services, Inc., the Queens office of The Tablet.  It also houses the Offices of Pastoral Planning  Family Life, Hispanic Ministry, Black Ministry, Respect Life and the Office of the Regional Bishop of Queens North as well as other agencies of the Diocese.