DI Management Home > Cryptography > Using flex and bison in MSVC++

Using flex and bison in MSVC++


Recommended reading

Join Amazon Student FREE Two-Day Shipping for College Students

Affiliate disclosure: we get a small commission for purchases made through the above links

This is part two of a series explaining how to convert example lex and yacc code from Lex & Yacc by Levine, Mason and Brown [1] to work on modern systems using flex and bison. Part one looked at compiling on a modern Linux system. This page explains how we made it compile without error on our Windows 7 system using Microsoft Visual Studio 2008 (MSVC++9). Note: This page is a bit out of date now, being written for MSVC 2008. See the note on Update for MSVC 2010+ below.

Most solutions we've seen for MSVC involve processing the flex and bison files on a Linux system like Cygwin and then copying the resulting .c and .h files across to MSVC to be compiled. Our solution here lets you put your .l (or .flex) and .y files directly into your MSVC project folder, edit them at will, and then process and compile it all with a single click.

YMMV. No doubt there are alternative and better ways to do it. Please let us know your suggestions.

 
Update for MSVC 2010+: We were surprised when we upgraded to MSVC 2013 and found that everything we described here just worked for our old MSVC 2008 projects. However, if you are starting with a later version of MSVC, you may find these instructions at Win flex-bison: setup custom build rules for Visual Studio 2010 and up of more use.
Book: There is a more up-to-date book than the one we originally used: Flex & Bison by John Levine [2] and a better download with more recent versions of Flex and Bison.
The changes to Flex and Bison mean that some of our original settings need to be modified. The graphics below may not reflect these changes.
30 May 2014: See the comments below from Alex Zhondin ("lexxmark") who is the owner of the winflexbison project at sourceforge. Thanks, Alex.

Contents

  1. Setting up flex and bison on Windows - a one-off
  2. Create a new project in MSVC++
  3. Add Custom Build Rules for flex and bison files - another one-off
  4. More changes to the Configuration Properties
  5. Add two wrapper files to your project
  6. Add the .l and .y files to your project and compile it
  7. A complete solution file
  8. A final twist for C++ and unistd.h
  9. Comments by the winflexbison project owner

Setting up flex and bison on Windows

Old instructions (superseded)

  1. Download and install the latest GNU binaries for Windows:
     
    • Flex for Windows.
    • Bison for Windows.
    • In each case download the "Setup" file for the "Complete package, except sources". As at March 2013, these should be flex-2.5.4a-1.exe (1.2 MB) and bison-2.4.1-setup.exe (3.7 MB). These files have not been updated for a while.
  2. Important: Install these packages in the new folder C:\GnuWin32, not the default folder with "Program Files" in the name. You must not have any spaces in the installation folder name or you will have problems. So when the setup program asks for the installation folder name, you must change it to a name without spaces. We suggest C:\GnuWin32.

A new and better download

  1. Download the latest Win flex-bison package from sourceforge win_flex_bison-latest.zip. The version we downloaded (648 kB, last updated 2013-11-22) contained win_flex version 2.5.37 and win_bison version 2.7.
  2. Unzip and copy the whole folder somewhere convenient. Do not store in a folder with spaces in the path. We use C:\GnuWin32\win_flex_bison. Make sure you include the subdirectory data which includes the m4sugar stuff.

    win_flex_bison directory

Create a new project in MSVC++

  1. In Microsoft Visual Studio create a new project: Visual C++ > Win32 > Win32 Console Application.

    Add new project

  2. Select the minimum application settings: Console application and Empty project.

    Win32 Application Settings

We will make some more changes to the configuration properties below after we've added our files. First we must do a one-off and add the custom build rules for flex and bison files.

Add Custom Build Rules for flex and bison files

You only need to do this step once and it is set for all subsequent projects.

  1. Right-click on the project in the Solution Explorer and select Custom Build Rules....

    Select Custom Build Rules...

  2. Click on New Rule File...

    New Rule File

  3. Set:

    • Display Name to Flex and Bison Tools
    • File Name to FlexBison.rules
    • and enter a Directory which you know will be backed up and be available for future projects.

    We will use this "rules" file in all our flex and bison projects. Save this file by clicking OK and then edit it again using Modify Rule File.

    New Rule File

  4. Add two build rules using Add Build Rule, one for Bison and one for Flex. Important: You should add Bison first.

    The important entries for Bison are:

    • Name = Bison
    • Command Line = C:\GnuWin32\bin\bison.exe -d -v -b y $(InputPath)
    • Command Line = C:\GnuWin32\win_flex_bison\win_bison.exe -d -v -b y $(InputPath)
    • File Extensions = *.y
    • Outputs = y.tab.c,y.tab.h

    Bison build rule

    The important entries for Flex are:

    • Name = Flex
    • Command Line = C:\GnuWin32\bin\flex.exe $(InputPath)
    • Command Line = C:\GnuWin32\win_flex_bison\win_flex.exe --wincompat $(InputPath)
    • File Extensions = *.l;*.flex
    • Outputs = lex.yy.c

    Note the --wincompat option for win_flex.

    Flex build rule

  5. Save all your changes. You should have Custom Build Rule Files that look like this for "Flex and Bison Tools":

    Bison build rule

More changes to the Configuration Properties

You need to make these changes for every project.

  1. Make sure the General Configuration Properties have Character Set = Not Set. We want to use the ANSI/ASCII character set, not Unicode.

    General configuration properties

  2. Go to the C++ Preprocessor screen (Configuration Properties > C/C++ > Preprocessor) and add the Preprocessor Definitions _CRT_SECURE_NO_DEPRECATE and _CRT_NONSTDC_NO_DEPRECATE.

    Preprocessor definitions

    These avoid more annoying but irrelevant warning messages from MSVC. Incidentally, you can do most of this setup as default: see MSVC++: Getting rid of that _CRT_SECURE_NO_DEPRECATE warning.

  3. Go to the C++ Advanced Properties screen (Configuration Properties > C/C++ > Advanced) and set Compile As = Compile as C Code (/TC) and Disable Specific Warnings = 4102.

    C++ Advanced properties

    The 4102 warning we disable is caused by the automatically-generated code [The label is defined but never referenced]. This is generally one of those warnings you can safely disable without fear of hiding any real errors you might make in your code.

    Update December 2013: The 4102 warning does not seem to happen with the new win_flex and win_bison. Instead we get [warning C4715: 'lookup' : not all control paths return a value]. If you wish, you can change the 4102 above to 4715 to suppress this message, but be aware that this is not such a safe warning to turn off!
  4. Go to the Linker Input screen (Configuration Properties > Linker > Input) and set Additional Dependencies = C:\GnuWIn32\lib\libfl.a C:\GnuWIn32\lib\liby.a. Use a space as separator between the two entries.
    Update December 2013: This step is probably unnecessary with the latest win_flex and win_bison, providing you provide your own main() and yyerror() functions, which you should do anyway. And you can avoid providing a yywrap() function by setting %option noyywrap in your flex file.

    Linker additional dependencies

    These library files (they are named ".a" but behave like ".lib" files) provide very minimal default functions. The library libfl.a provides yywrap() and main(), and liby.a provides yyerror() and its own version of main(). In practice, you are probably better off providing your own versions. If you provide your own functions, you don't need to refer to these two libraries.

Add two wrapper files to your project

Add these two one-liner wrapper files wrapper-flex.c and wrapper-bison.c to your project.


/* wrapper-flex.c */
#include "lex.yy.c"

/* wrapper-bison.c */
#include "y.tab.c"

These make it easier to handle the frequently-changing lex.yy.c and y.tab.c files. Note this relies on bison using the "-b y" option (see the note below). The disadvantage of this technique is that you must always keep each set of project files in separate folders.

Add the .l and .y files to your project and compile it

Finally, we can add our flex and bison files to the MSVC project. In this case we add our modified ch3-03.l and ch3-03.y files and flex_memory_fix.h (zipped files here, 2 kB). When we add these files, with the .l and .y extensions, MSVC should prompt us to use the FlexBison.rules custom build rules we made earlier.

Solution Explorer

Keep all these files (ch3-03.l, ch3-03.y, flex_memory_fix.h, wrapper-bison.c, wrapper-flex.c) in the same project folder. The files (lex.yy.c, y.output, y.tab.c, y.tab.h) are created automatically, so don't mess with them.

Files

Update December 2013: The file flex_memory_fix.h is no longer needed with the latest version of win_bison.

If all has gone correctly, you should be able to compile this project. (It might take two goes the first time.) On our VS2008 system, we get this output:

1>------ Rebuild All started: Project: flexbison-test, Configuration: Debug Win32 ------
1>Deleting intermediate and output files for project 'flexbison-test', configuration 'Debug|Win32'
1>Running bison
1>Running flex
1>Compiling...
1>wrapper-bison.c
1>wrapper-flex.c
1>Generating Code...
1>Compiling manifest to resources...
1>Microsoft (R) Windows (R) Resource Compiler Version 6.1.6723.1
1>Copyright (C) Microsoft Corporation.  All rights reserved.
1>Linking...
1>LINK : C:\!Data\DotNet\Cpp\flexbison-test\Debug\flexbison-test.exe not found or not built by the last incremental link; performing full link
1>Embedding manifest...
1>Microsoft (R) Windows (R) Resource Compiler Version 6.1.6723.1
1>Copyright (C) Microsoft Corporation.  All rights reserved.
1>Build log was saved at "file://c:\!Data\DotNet\Cpp\flexbison-test\flexbison-test\Debug\BuildLog.htm"
1>flexbison-test - 0 error(s), 0 warning(s)
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========

Note the "Running bison" and "Running flex" lines in the output. These are the values in "Execution Description" we set when we created the build rules. Note it is important that bison runs first before flex.

A note about the "-b y" option for bison

The -b y option we set for win_bison makes sure the names of the output files are always y.tab.h and y.tab.c (instead of foo.tab.h where foo.y is the name of your bison file). This is convenient for us in the setup above. However, you may need to change your flex foo.l file if it refers to a file like foo.tab.h and make that y.tab.h

/* foo.l */
//...
%option noyywrap nodefault yylineno
%{
#include "foo.h"
#include "foo.tab.h"  #include "y.tab.h"
%}

A final twist for C++ and unistd.h

There is one more twist to all the above if you actually want to compile C++ code with flex and bison (as opposed to C code with the /TC option). If you change the C/C++ > Advanced > Compile As option back to Compile as C++ Code (/TP), then you will run into problems where the automatically-generated source code looks for but cannot find unistd.h.

As far as we know, this include file is not available in MSVC/Windows. A copy that works on Windows is in the GNU folder C:\GnuWin32\include. One fix is to add C:\GnuWin32\include to the VC++ Directories Include Files in Tools > Options > Projects and Solutions.

References

Comments by the winflexbison project owner

Hello everyone!
I've just seen your article at http://www.di-mgt.com.au/flex_and_bison_in_msvc.html and have two comments.

First of all I'm owner of the winflexbison project at sourceforge and glad to see my project is useful for you.

  1. At the end of the article you noted about "unistd.h" include problem. I thought I resoled this problem with "--wincompat" key. You can see little explanation in UNISTD_ERROR.readme file in win_flex_bison root folder. If there is still problems with "unistd.h" include, please let me know - it should be fixed.
  2. Also you noted about warning C4715 "not all control paths return a value" in generated source files. I think I should fix it too.
Alex | lexxmark - Mon Mar 24 19:55:14 2014 GMT

Hi Alex, Great to hear from you. Your project is really good.

Some of the comments on the page refer to my attempts with the earlier version. I think the unistd.h problem related to that. To be honest I do everything in plain ANSI C, and never use the C++ options, so I wouldn't have noticed the effect of using the --wincompat option on the new version. I'll fix up the comment in due course. Our ISP is moving the site over to a new server at the moment and taking their time over it, so we cannot make any changes for the time being.

The other warning about control paths is mildly irritating, and as a rule I don't like turning off warnings. If you could fix it it would be great. But it's not critical.

The product is really great. Please keep up the good work.

Dave | Moderator - Mar 25 05:40 AM 2014

Recently I was asked to add custom build rules for VS2010 and upper to winflexbison package (see http://sourceforge.net/p/winflexbison/discussion/general/thread/6c305702). So you can use those ruleset if you will migrate from VS2008.

During creation rules I've noticed a little problem with invoking win_flex.exe or win_bison.exe tools from VS build system. May be you will be interested. The invoking working directory is not the same as *.y or *.l files placed in. So generated C code have #line directives with absolute paths to source grammar file. It can be critical if you place generated files under version control system and on different machines you have different paths.

I hope I describe the problem more or less clear. To fix it I had to change simple invoke command like:

win_flex.exe $(InputPath)

to more magic one:

start /B /WAIT /D "%(RootDir)%(Directory)" win_flex.exe $(InputPath)

This will setup correct working directory and #line directives will be relative then. Though for learning purposes this is not so critical. Best wishes,

Alex | lexxmark - Tue, 01 Apr 2014 12:55:56

Contact

To comment on this page or to contact us, please send us a message. If you send us a relevant comment for this page, we'll post it here. Just mention "flex_and_bison_in_msvc" in your message.

This page first published 22 March 2013. Last updated 12 July 2016.