The M.A.R.V.I.N. (Mobile, Autonomous Robot with Visually Influenced Navigation) attempts to use visual recognition to locate its position in its environment and navigate accordingly. The robot uses a number of technologies to accomplish this. The robot uses the combination of the laptop and webcam to act as an information gathering unit. A webpage interface acts as a cross-platform distributed graphical user interface, or dgui. Finally, a supercomputer, a machine in the physics department, performs the image analysis routines. The following is detailed description of each part.
The laptop and webcam combo perform multiple functions. The laptop runs the ARIA daemon, the NetListenerTask, and the camdriver objects. The NetListenerTask accepts socket connections from local or distributed clients. This flexibility allows for development and debugging of the various components on the laptop, while at the same time allowing for easy rollout of the final project. The camdriver is also socket driven and allows for the image aquisition processes to capture an image from the webcam. A lot of time was spent in this area to insure the quality of the picture capturing process.
The distributed graphical user interface was implemented as an afterthought, therefore it doesn't appear in our design documentation or in the object model. Its main purpose is to monitor the robots behavior and direct its behavior. The dgui is served by an Apache webserver running on the same machine that does the image analysis. The webserver can interpret php, a general purpose scripting language that can be embedded in html, and uses this functionality to connect to the robot using sockets. Once connected the dgui can communicate with the NetListenerTask, a object that sends commands to the robot, such as move forwards, turn right or left, etc. The dgui also talks to the camdriver and aquires images when requested by the user. These images are then processed by the image analysis routines. In the future, the dgui will be unnecessary once sufficient confidence in the robots behavior has been gained.
The image analysis can be considered the bulk of the project. The program, consisting of several objects and various subroutines, runs on a supercomputer in the physics department. The computer runs screaming dual palimino processors and contains a beastly one gig of RAM. This massive computing power is put to good use when running the image analysis routines. Don't be fooled by the quick processing time and quick return of a processed image. On most computers, the complicated algorithms and multiple calculations would incure a heavy price on the processor.
The Aria Daemon is a complex multi-threaded application. There is a thread for accepting commands over the network, there is a thread controlling the robot, and there are threads that a spawned internally within the aria library. For this reason, coverage is done by using the ariad over an extended period of time.
Although each individual thread is fairly simplistic, the interaction between threads (and locking of the robot) and callback functions make it hard to do a trace by hand. Therefore testing is done by running ariad over a long period of time and storing the resulting trace data to a file (using compile time option -pg and -a) and then using gprof/tcov to verify that every line has been hit. Although this gives us C0 coverage, this is the only code that we did not have C1 coverage for.
The camera driver is actually a fairly simplistic, straight procedural program. There is litte room for different execution paths except for initialization. The program simply sits and waits on a socket. As soon as someone connects an image is take from the camera and the results are sent through the socket (whether the other side wants it or not). It then closes the socket and waits for another connection. C0 coverage and C1 coverage is trivially shown using gprof/tcov.
C0 coverage was achieved in my code by simply outputting a message as each statement is executed. My code was concise enough that this could be done by hand. The following macro was placed after every semicolon in my code with an appropriate node label:
#define NODE(n) printf(ôNode n executed.\nö);
while(x != img.x_boundaries.end())
{ printf(ô%d %dö,**x,**x);NODE(a)
printf(ô%d %d\n,**y,**y2);NODE(b)
à
I then examined the output to see if every node had been executed at least once. Given a proper image to process, C0 coverage is achieved for ImageSegregate.
The FindNumber class basically compares an image taken as an argument and a set of ideal numbers. The ideal numbers are each initialized with a switch statement. Once the object has been initialized, the findNumber function may be called. The findNumber function operates by executing a for-loop 10 times, once for each digit. In the for-loop each image is compared to the size of the "ideal" number pixel by pixel. Once this has been done, a ratio of similar pixels versus total pixels is calculated and stored in an array. Finally, each ratio is examined and the maximum ratio is pulled out and the index of the ratio in the array equates to the number found. The following is a printout of a C0 and C1 test coverage. The test is performed using mytest.cpp, found in the code section.
Initialize zero Initialize one Initialize two Initialize three Initialize four Initialize five Initialize six Initialize seven Initialize eight Initialize nine Begin findNumber routine iteration 0 of comparison loop resize image Found similar RED Found similar BLUE Found similar GREEN Count plus three ... Found similar RED Found similar BLUE Found similar GREEN Count plus three iteration 1 of comparison loop ... iteration 2 of comparison loop ... iteration 3 of comparison loop ... iteration 4 of comparison loop ... iteration 5 of comparison loop ... iteration 6 of comparison loop ... iteration 7 of comparison loop ... iteration 8 of comparison loop ... iteration 9 of comparison loop ... Finished comparing numbers Found a higher maxval in ratios[0] Found a higher maxval in ratios[1] 1 1.000000 Done!
#ifndef ACTIONGO_H
#define ACTIONGO_H
#include "Aria.h"
class ActionGo : public ArAction
{
public:
// constructor, sets myMaxSpeed and myStopDistance
ActionGo();
// destructor, its just empty, we don't need to do anything
virtual ~ActionGo(void) {};
// fire, this is what the resolver calls to figure out what this action wants
virtual ArActionDesired *fire(ArActionDesired currentDesired);
// sets the robot pointer, also gets the sonar device
virtual void setRobot(ArRobot *robot);
virtual void setSpeed(double newspeed);
protected:
// this is to hold the sonar device form the robot
ArRangeDevice *mySonar;
// what the action wants to do
ArActionDesired myDesired;
// maximum speed
double speed;
double getVel; // robot too unreliable
double avgVel[10]; int avgPos;
};
#endif
#ifndef ACTIONTURN_H
#define ACTIONTURN_H
#include "Aria.h"
class ActionTurn : public ArAction
{
public:
// constructor, sets the turnThreshold, and turnAmount
ActionTurn();
// destructor, its just empty, we don't need to do anything
virtual ~ActionTurn(void) {};
// fire, this is what the resolver calls to figure out what this action wants
virtual ArActionDesired *fire(ArActionDesired currentDesired);
// sets the robot pointer, also gets the sonar device
virtual void setRobot(ArRobot *robot);
virtual void setTurnAmount(double newTurnAmount);
protected:
// this is to hold the sonar device form the robot
ArRangeDevice *mySonar;
// what the action wants to do
ArActionDesired myDesired;
double myTurnAmount;
};
#endif
#ifndef NETLISTENERTASK_H
#define NETLISTENERTASK_H
#include "Aria.h"
#include "ActionGo.h"
#include "ActionTurn.h"
class NetListenerTask : public ArASyncTask {
public:
NetListenerTask(ArRobot *arobot);
~NetListenerTask(void) {};
int startListening(int port);
virtual void *runThread(void *arg);
private:
ArSocket serverSock, clientSock;
ArRobot* robot;
ActionGo* go;
ActionTurn* myturn;
};
#endif
#include "ActionGo.h"
/*
This is the constructor, note the use of constructor chaining with the
ArAction... also note how it uses setNextArgument, which makes it so that
other things can see what parameters this action has, and set them.
It also initializes the classes variables.
*/
ActionGo::ActionGo() : ArAction("Go")
{
mySonar = NULL;
speed=0;
getVel=0;
for(avgPos=0;avgPos<10;avgPos++) avgVel[avgPos]=0;
avgPos=0;
}
/*
Sets the myRobot pointer (all setRobot overloaded functions must do this),
finds the sonar device from the robot, and if the sonar isn't there,
then it deactivates itself.
*/
void ActionGo::setRobot(ArRobot *robot)
{
myRobot = robot;
mySonar = myRobot->findRangeDevice("sonar");
if (mySonar == NULL)
deactivate();
}
void ActionGo::setSpeed(double newspeed)
{
speed=newspeed;
}
/*
This fire is the whole point of the action.
*/
ArActionDesired *ActionGo::fire(ArActionDesired currentDesired)
{
double range, curVel, change;
// reset the actionDesired (must be done)
myDesired.reset();
// if the sonar is null we can't do anything, so deactivate
if (mySonar == NULL)
{
deactivate();
return NULL;
}
// curVel = myRobot->getVel(); //getVel;
avgPos=(avgPos+1)%10;
avgVel[avgPos] = myRobot->getVel();
// cout << "values " << avgPos;
for(int i=0;i<10;i++) {
curVel+=avgVel[i];
// cout << " " << avgVel[i];
}
// cout << endl;
curVel=curVel/10;
// cout << endl << curVel << " " << speed << endl;
if(curVel<10 && curVel>-10) curVel=0;
change=speed-curVel;
if(change>50) change=50;
if(change<-50) change=-50;
curVel+=change;
// cout << "setting speed to " << curVel << endl;
//getVel=curVel;
myDesired.setVel(curVel);
// return a pointer to the actionDesired, so resolver knows what to do
return &myDesired;
}
#include "ActionTurn.h"
/*
This is the constructor, note the use of constructor chaining with the
ArAction... also note how it uses setNextArgument, which makes it so that
other things can see what parameters this action has, and set them.
It also initializes the classes variables.
*/
ActionTurn::ActionTurn() : ArAction("Turn")
{
mySonar=NULL;
myTurnAmount = 0;
}
/*
Sets the myRobot pointer (all setRobot overloaded functions must do this),
finds the sonar device from the robot, and if the sonar isn't there,
then it deactivates itself.
*/
void ActionTurn::setRobot(ArRobot *robot)
{
myRobot = robot;
mySonar = myRobot->findRangeDevice("sonar");
if (mySonar == NULL)
deactivate();
}
void ActionTurn::setTurnAmount(double newTurnAmount) {
myTurnAmount=newTurnAmount;
}
/*
This is the guts of the action.
*/
ArActionDesired *ActionTurn::fire(ArActionDesired currentDesired)
{
double leftRange, rightRange;
// reset the actionDesired (must be done)
myDesired.reset();
// if the sonar is null we can't do anything, so deactivate
if (mySonar == NULL)
{
deactivate();
return NULL;
}
/*
// Get the left readings and right readings off of the sonar
leftRange = (mySonar->currentReadingPolar(0, 100) -
myRobot->getRobotRadius());
rightRange = (mySonar->currentReadingPolar(-100, 0) -
myRobot->getRobotRadius());
*/
// if neither left nor right range is within the turn threshold,
// reset the turning variable and don't turn
if (myTurnAmount) {
myDesired.setDeltaHeading(-myTurnAmount);
myTurnAmount=0;
}
// return a pointer to the actionDesired, so resolver knows what to do
return &myDesired;
}
#include "NetListenerTask.h"
int numofcommands=6;
char* netcommandstrs[]={"stop","move","setDeltaHeading","turn","setVel","say"};
enum netcommands { stop, move, setDeltaHeading, turn, setVel, say };
NetListenerTask::NetListenerTask(ArRobot *arobot) {
robot=arobot;
robot->lock();
go=new ActionGo();
myturn=new ActionTurn();
robot->addAction(go, 50);
robot->addAction(myturn, 50);
robot->unlock();
cout << "Creating NetListenerTask" << endl;
}
/* this sholud be called before creating the thread to make sure
* everything is okay */
int NetListenerTask::startListening(int port) {
while(!serverSock.open(port, ArSocket::TCP)) {
cout << "Failed to open port " << port
<< ": " << serverSock.getErrorStr().c_str() << endl;
usleep(1000000);
}
cout << "Opened port " << port << endl;
return(port);
}
void *NetListenerTask::runThread(void *arg) {
char buffer[1024];
char* argpos;
long arg_int;
double arg_double;
FILE *sayf=0;
cout << "NetListenerTask Running" << endl;
while(serverSock.accept(&clientSock)) {
cout << "Client has connected" << endl;
// here we read up to 1024 characters and wait up to 500ms
int read=clientSock.read(buffer,sizeof(buffer),500);
if(read>=500) read=499;
buffer[read]=0;
cout << "Read Buffer: " << buffer << endl;
int foundcommand=0;
argpos=0;
for(;foundcommand<numofcommands;foundcommand++) {
if(strlen(buffer)>=strlen(netcommandstrs[foundcommand]) &&
strncmp(netcommandstrs[foundcommand],buffer,strlen(netcommandstrs[foundcommand]))==0) {
argpos=buffer+strlen(netcommandstrs[foundcommand]);
break;
}
}
robot->lock();
switch (foundcommand) {
case stop:
//robot->stop();
go->setSpeed(0);
myturn->setTurnAmount(0);
break;
case setVel:
arg_double=strtod(argpos,&argpos);
cout << "setVel " << arg_double << endl;
go->setSpeed(arg_double);
break;
case move:
/*
arg_int=strtol(argpos,&argpos,10);
cout << "move " << arg_int << endl;
robot->move(arg_int);
*/
cout << "ignoring move command" << endl;
break;
case turn:
case setDeltaHeading:
arg_double=strtod(argpos,&argpos);
cout << "setDeltaHeading " <<arg_double<<endl;
//robot->setDeltaHeading(arg_double);
myturn->setTurnAmount(arg_double);
break;
case say:
if(fork()==0) {
sayf=popen("/usr/bin/festival_client","w");
if(sayf>0) {
fprintf(sayf,"(SayText \"");
fprintf(sayf,"%s",argpos);
fprintf(sayf,"\")\n");
pclose(sayf);
} else {
cout << "error opening fest" << endl;
}
exit(0);
}
break;
default:
cout << "Unknown Command\n";
break;
};
robot->unlock();
cout << "Client has disconnected" << endl;
clientSock.close();
}
cout << "Something bad happened while accepting connection" << serverSock.getErrorStr().c_str() << endl;
serverSock.close();
cout << "Leaving NetListenerTask Thread" << endl;
return NULL; // close thread
}
#include <iostream.h>
#include "Aria.h"
#include "NetListenerTask.h"
int main(void) {
ArTcpConnection tcpConn; // simulator
ArSerialConnection serConn; // robot connection
ArRobot robot;
ArSonarDevice sonar;
ArKeyHandler keyHandler; // handle ESC
Aria::init(Aria::SIGHANDLE_THREAD);
Aria::setKeyHandler(&keyHandler);
robot.attachKeyHandler(&keyHandler);
tcpConn.setPort();
cout << "Trying to connect to simulator first" << endl;
if(tcpConn.openSimple()) {
cout << "Connecting to simulator" << endl;
robot.setDeviceConnection(&tcpConn);
} else {
serConn.setPort();
cout << "Connecting to serial (physical robot)" << endl;
robot.setDeviceConnection(&serConn);
}
robot.addRangeDevice(&sonar);
if (!robot.blockingConnect()) {
cout << "Could not connect to robot... exiting" << endl;
Aria::shutdown();
return 1;
}
// limit speed
robot.setMaxTransVel(200);
// enable motors
robot.comInt(ArCommands::ENABLE, 1);
ArActionLimiterForwards limiter("forward near",250,600,200);
ArActionLimiterBackwards backwardsLimiter("backwards Limiter",-250,-600,-200);
robot.addAction(&limiter,85);
robot.addAction(&backwardsLimiter,85);
// run robot in it's own thread
robot.runAsync(false);
NetListenerTask netlistener(&robot);
if (netlistener.startListening(5401)<0) {
cerr << "Error starting listener" << endl;
return -1;
}
netlistener.create();
// wait for all threads to quite
ArThread::joinAll();
Aria::uninit();
Aria::shutdown();
return 0;
}
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <linux/types.h>
#include <linux/videodev.h>
#include "v4l.h"
#define min(a,b) ((a) < (b) ? (a) : (b))
#define max(a,b) ((a) > (b) ? (a) : (b))
/*
* set the input and norm for the video4linux device
*/
int
v4l_set_input (int fd, int input, int norm)
{
struct video_channel vid_chnl;
if (input != INPUT_DEFAULT || norm != NORM_DEFAULT) {
if (vid_chnl.channel != INPUT_DEFAULT)
vid_chnl.channel = input;
else
vid_chnl.channel = 0;
vid_chnl.norm = -1;
if (ioctl (fd, VIDIOCGCHAN, &vid_chnl) == -1) {
perror ("ioctl (VIDIOCGCHAN)");
return -1;
} else {
if (input != 0)
vid_chnl.channel = input;
if (norm != NORM_DEFAULT)
vid_chnl.norm = norm;
if (ioctl (fd, VIDIOCSCHAN, &vid_chnl) == -1) {
perror ("ioctl (VIDIOCSCHAN)");
return -1;
}
}
}
return 0;
}
/*
* check the size and readjust if necessary
*/
int
v4l_check_size (int fd, int *width, int *height)
{
struct video_capability vid_caps;
if (ioctl (fd, VIDIOCGCAP, &vid_caps) == -1) {
perror ("ioctl (VIDIOCGCAP)");
return -1;
}
/* readjust if necessary */
if (*width > vid_caps.maxwidth || *width < vid_caps.minwidth) {
*width = min (*width, vid_caps.maxwidth);
*width = max (*width, vid_caps.minwidth);
fprintf (stderr, "readjusting width to %d\n", *width);
}
if (*height > vid_caps.maxheight || *height < vid_caps.minheight) {
*height = min (*height, vid_caps.maxheight);
*height = max (*height, vid_caps.minheight);
fprintf (stderr, "readjusting height to %d\n", *height);
}
return 0;
}
/*
* check the requested palette and adjust if possible
* seems not to work :-(
*/
int
v4l_check_palette (int fd, int *palette)
{
struct video_picture vid_pic;
if (!palette)
return -1;
if (ioctl (fd, VIDIOCGPICT, &vid_pic) == -1) {
perror ("ioctl (VIDIOCGPICT)");
return -1;
}
vid_pic.palette = *palette;
if (ioctl (fd, VIDIOCSPICT, &vid_pic) == -1) {
/* try YUV420P
*/
fprintf (stderr, "failed\n");
vid_pic.palette = *palette = VIDEO_PALETTE_YUV420P;
if (ioctl (fd, VIDIOCSPICT, &vid_pic) == -1) {
perror ("ioctl (VIDIOCSPICT) to YUV");
/* ok, try grayscale..
*/
vid_pic.palette = *palette = VIDEO_PALETTE_GREY;
if (ioctl (fd, VIDIOCSPICT, &vid_pic) == -1) {
perror ("ioctl (VIDIOCSPICT) to GREY");
return -1;
}
}
}
return 0;
}
/*
* check if driver supports mmap'ed buffer
*/
int
v4l_check_mmap (int fd, int *size)
{
struct video_mbuf vid_buf;
if (ioctl (fd, VIDIOCGMBUF, &vid_buf) == -1) {
return -1;
}
if (size)
*size = vid_buf.size;
return 0;
}
/*
* Turn a YUV4:2:0 block into an RGB block
*
* Video4Linux seems to use the blue, green, red channel
* order convention-- rgb[0] is blue, rgb[1] is green, rgb[2] is red.
*
* Color space conversion coefficients taken from the excellent
* http://www.inforamp.net/~poynton/ColorFAQ.html
* In his terminology, this is a CCIR 601.1 YCbCr -> RGB.
* Y values are given for all 4 pixels, but the U (Pb)
* and V (Pr) are assumed constant over the 2x2 block.
*
* To avoid floating point arithmetic, the color conversion
* coefficients are scaled into 16.16 fixed-point integers.
* They were determined as follows:
*
* double brightness = 1.0; (0->black; 1->full scale)
* double saturation = 1.0; (0->greyscale; 1->full color)
* double fixScale = brightness * 256 * 256;
* int rvScale = (int)(1.402 * saturation * fixScale);
* int guScale = (int)(-0.344136 * saturation * fixScale);
* int gvScale = (int)(-0.714136 * saturation * fixScale);
* int buScale = (int)(1.772 * saturation * fixScale);
* int yScale = (int)(fixScale);
*/
/* LIMIT: convert a 16.16 fixed-point value to a byte, with clipping. */
#define LIMIT(x) ((x)>0xffffff?0xff: ((x)<=0xffff?0:((x)>>16)))
/*
*/
static inline void
v4l_copy_420_block (int yTL, int yTR, int yBL, int yBR, int u, int v,
int rowPixels, unsigned char * rgb, int bits)
{
const int rvScale = 91881;
const int guScale = -22553;
const int gvScale = -46801;
const int buScale = 116129;
const int yScale = 65536;
int r, g, b;
g = guScale * u + gvScale * v;
r = rvScale * v;
b = buScale * u;
yTL *= yScale; yTR *= yScale;
yBL *= yScale; yBR *= yScale;
if (bits == 24) {
/* Write out top two pixels */
rgb[0] = LIMIT(b+yTL); rgb[1] = LIMIT(g+yTL); rgb[2] = LIMIT(r+yTL);
rgb[3] = LIMIT(b+yTR); rgb[4] = LIMIT(g+yTR); rgb[5] = LIMIT(r+yTR);
/* Skip down to next line to write out bottom two pixels */
rgb += 3 * rowPixels;
rgb[0] = LIMIT(b+yBL); rgb[1] = LIMIT(g+yBL); rgb[2] = LIMIT(r+yBL);
rgb[3] = LIMIT(b+yBR); rgb[4] = LIMIT(g+yBR); rgb[5] = LIMIT(r+yBR);
} else if (bits == 16) {
/* Write out top two pixels */
rgb[0] = ((LIMIT(b+yTL) >> 3) & 0x1F) | ((LIMIT(g+yTL) << 3) & 0xE0);
rgb[1] = ((LIMIT(g+yTL) >> 5) & 0x07) | (LIMIT(r+yTL) & 0xF8);
rgb[2] = ((LIMIT(b+yTR) >> 3) & 0x1F) | ((LIMIT(g+yTR) << 3) & 0xE0);
rgb[3] = ((LIMIT(g+yTR) >> 5) & 0x07) | (LIMIT(r+yTR) & 0xF8);
/* Skip down to next line to write out bottom two pixels */
rgb += 2 * rowPixels;
rgb[0] = ((LIMIT(b+yBL) >> 3) & 0x1F) | ((LIMIT(g+yBL) << 3) & 0xE0);
rgb[1] = ((LIMIT(g+yBL) >> 5) & 0x07) | (LIMIT(r+yBL) & 0xF8);
rgb[2] = ((LIMIT(b+yBR) >> 3) & 0x1F) | ((LIMIT(g+yBR) << 3) & 0xE0);
rgb[3] = ((LIMIT(g+yBR) >> 5) & 0x07) | (LIMIT(r+yBR) & 0xF8);
}
}
/*
*/
static inline void
v4l_copy_422_block (int yTL, int yTR, int u, int v,
int rowPixels, unsigned char * rgb, int bits)
{
const int rvScale = 91881;
const int guScale = -22553;
const int gvScale = -46801;
const int buScale = 116129;
const int yScale = 65536;
int r, g, b;
g = guScale * u + gvScale * v;
r = rvScale * v;
b = buScale * u;
yTL *= yScale; yTR *= yScale;
if (bits == 24) {
/* Write out top two pixels */
rgb[0] = LIMIT(b+yTL); rgb[1] = LIMIT(g+yTL); rgb[2] = LIMIT(r+yTL);
rgb[3] = LIMIT(b+yTR); rgb[4] = LIMIT(g+yTR); rgb[5] = LIMIT(r+yTR);
} else if (bits == 16) {
/* Write out top two pixels */
rgb[0] = ((LIMIT(b+yTL) >> 3) & 0x1F) | ((LIMIT(g+yTL) << 3) & 0xE0);
rgb[1] = ((LIMIT(g+yTL) >> 5) & 0x07) | (LIMIT(r+yTL) & 0xF8);
rgb[2] = ((LIMIT(b+yTR) >> 3) & 0x1F) | ((LIMIT(g+yTR) << 3) & 0xE0);
rgb[3] = ((LIMIT(g+yTR) >> 5) & 0x07) | (LIMIT(r+yTR) & 0xF8);
}
}
/*
* convert a YUV420P to a rgb image
*/
int
v4l_yuv420p2rgb (unsigned char *rgb_out, unsigned char *yuv_in,
int width, int height, int bits)
{
const int numpix = width * height;
const unsigned int bytes = bits >> 3;
int h, w, y00, y01, y10, y11, u, v;
unsigned char *pY = yuv_in;
unsigned char *pU = pY + numpix;
unsigned char *pV = pU + numpix / 4;
unsigned char *pOut = rgb_out;
if (!rgb_out || !yuv_in)
return -1;
for (h = 0; h <= height - 2; h += 2) {
for (w = 0; w <= width - 2; w += 2) {
y00 = *(pY);
y01 = *(pY + 1);
y10 = *(pY + width);
y11 = *(pY + width + 1);
u = (*pU++) - 128;
v = (*pV++) - 128;
v4l_copy_420_block (y00, y01, y10, y11, u, v, width, pOut, bits);
pY += 2;
pOut += bytes << 1;
}
pY += width;
pOut += width * bytes;
}
return 0;
}
/*
* convert a YUV422P to a rgb image
*/
int
v4l_yuv422p2rgb (unsigned char *rgb_out, unsigned char *yuv_in,
int width, int height, int bits)
{
const int numpix = width * height;
const unsigned int bytes = bits >> 3;
int h, w, y00, y01, u, v;
unsigned char *pY = yuv_in;
unsigned char *pU = pY + numpix;
unsigned char *pV = pU + numpix / 2;
unsigned char *pOut = rgb_out;
if (!rgb_out || !yuv_in)
return -1;
for (h = 0; h < height; h += 1) {
for (w = 0; w <= width - 2; w += 2) {
y00 = *(pY);
y01 = *(pY + 1);
u = (*pU++) - 128;
v = (*pV++) - 128;
v4l_copy_422_block (y00, y01, u, v, width, pOut, bits);
pY += 2;
pOut += bytes << 1;
}
//pY += width;
//pOut += width * bytes;
}
return 0;
}
#ifndef __V4L_H__ #define __V4L_H__ #define INPUT_DEFAULT -1 #define NORM_DEFAULT -1 #ifndef TRUE #define TRUE 1 #endif #ifndef FALSE #define FALSE 0 #endif int v4l_set_input (int fd, int input, int norm); int v4l_check_size (int fd, int *width, int *height); int v4l_check_palette (int fd, int *palette); int v4l_check_mmap (int fd, int *size); int v4l_yuv420p2rgb (unsigned char *, unsigned char *, int, int, int); int v4l_yuv422p2rgb (unsigned char *, unsigned char *, int, int, int); #endif
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <getopt.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/time.h> /* gettimeofday() */
#include <fcntl.h>
#include <unistd.h>
#include <linux/types.h>
#include <linux/videodev.h>
#include "v4l.h"
#include<sys/types.h>
#include<sys/socket.h>
#include<netinet/in.h>
#include<unistd.h>
#include<stdlib.h>
#include<stdio.h>
#define DEF_WIDTH 352 /* default width */
#define DEF_HEIGHT 240 /* default height */
#define QUAL_DEFAULT 80
char *basename (const char *s);
/* globals
*/
/*
*/
void
usage (char *pname)
{
fprintf (stderr,
"VidCat, Version %s\n"
"Usage: %s <options>\n"
" -b make a raw PPM instead of an ASCII one\n"
" -d <device> video device (default: "VIDEO_DEV")\n"
" -f {ppm|jpeg|png|yuv4mpeg} output format of the image\n"
" -g greayscale instead of color\n"
" -i {tv|comp1|comp2|s-video} which input channel to use\n"
" -l loop on, doesn't make sense in most cases\n"
" -n {pal|ntsc|secam} select video norm\n"
" -o <file> write output to file instead of stdout\n"
" -p c|g|y|Y videopalette to use\n"
" -q <quality> only for jpeg: quality setting (1-100,"
" default: %d)\n"
" -s NxN define size of the output image (default:"
" %dx%d)\n"
"Example: vidcat | xsetbg stdin\n",
VERSION, (char*)basename(pname), QUAL_DEFAULT, DEF_WIDTH, DEF_HEIGHT);
exit (1);
}
/*
* read rgb image from v4l device
* return: mmap'ed buffer and size
*/
char *
get_image (int dev, int width, int height, int palette ,int *size)
{
struct video_mbuf vid_buf;
struct video_mmap vid_mmap;
char *map, *convmap;
int len;
int bytes = 3;
if (ioctl (dev, VIDIOCGMBUF, &vid_buf) == -1) {
/* to do a normal read()
*/
struct video_window vid_win;
if (ioctl (dev, VIDIOCGWIN, &vid_win) != -1) {
vid_win.width = width;
vid_win.height = height;
if (ioctl (dev, VIDIOCSWIN, &vid_win) == -1) {
perror ("ioctl(VIDIOCSWIN)");
return (NULL);
}
}
map = malloc (width * height * bytes);
len = read (dev, map, width * height * bytes);
if (len <= 0) {
free (map);
return (NULL);
}
*size = 0;
}
map = mmap (0, vid_buf.size, PROT_READ|PROT_WRITE,MAP_SHARED,dev,0);
if ((unsigned char *)-1 == (unsigned char *)map) {
perror ("mmap()");
return (NULL);
}
vid_mmap.format = palette;
vid_mmap.frame = 0;
vid_mmap.width = width;
vid_mmap.height = height;
if (ioctl (dev, VIDIOCMCAPTURE, &vid_mmap) == -1) {
perror ("VIDIOCMCAPTURE");
fprintf (stderr, "args: width=%d height=%d palette=%d\n",
vid_mmap.width, vid_mmap.height, vid_mmap.format);
munmap (map, vid_buf.size);
return (NULL);
}
if (ioctl (dev, VIDIOCSYNC, &vid_mmap.frame) == -1) {
perror ("VIDIOCSYNC");
munmap (map, vid_buf.size);
return (NULL);
}
*size = vid_buf.size;
return (map);
}
/*
* write ppm image to stdout / file
*/
void
put_image_ppm (int out, char *image, int width, int height, int binary)
{
int x, y, ls=0;
char buffer[1024];
unsigned char *p = (unsigned char *)image;
if (!binary) {
sprintf(buffer,"P3\n%d %d\n%d\n", width, height, 255);
send(out,buffer,strlen(buffer),MSG_NOSIGNAL);
//fprintf (out, "P3\n%d %d\n%d\n", width, height, 255);
for (x = 0; x < width; x++) {
for (y = 0; y < height; y++) {
sprintf (buffer, "%03d %03d %03d ", p[2], p[1], p[0]);
send(out,buffer,strlen(buffer),MSG_NOSIGNAL);
//fprintf (out, "%03d %03d %03d ", p[2], p[1], p[0]);
p += 3;
if (ls++ > 4) {
send(out,"\n",1,MSG_NOSIGNAL);
//fprintf (out, "\n");
ls = 0;
}
}
}
//fprintf (out, "\n");
send(out,"\n",1,MSG_NOSIGNAL);
} else {
unsigned char buff[3];
sprintf (buffer, "P6\n%d %d\n%d\n", width, height, 255);
send(out,buffer,strlen(buffer),MSG_NOSIGNAL);
//fprintf (out, "P6\n%d %d\n%d\n", width, height, 255);
for (x = 0; x < width * height; x++) {
buff[0] = p[2];
buff[1] = p[1];
buff[2] = p[0];
//fwrite (buff, 1, 3, out);
send(out,buff,3,MSG_NOSIGNAL);
p += 3;
}
}
//fflush (out);
}
/*
* main()
*/
int
main (int argc, char *argv[])
{
int width = DEF_WIDTH, height = DEF_HEIGHT, size, dev = -1, c;
char *image, *device = VIDEO_DEV, *file = NULL;
int max_try = 5; /* we try 5 seconds/times to open the device */
int quality = QUAL_DEFAULT; /* default jpeg quality setting */
int input = INPUT_DEFAULT; /* this means take over current device settings*/
int norm = NORM_DEFAULT;
int binary = 1;
int one = 1; // for setsockopt
int loop=0,num=0;
int palette = VIDEO_PALETTE_RGB24;
FILE *out = stdout;
int create_socket,new_socket,addrlen;
struct sockaddr_in address;
// start sock code
//
if((create_socket = socket(AF_INET,SOCK_STREAM,0))<0) {
fprintf(stderr,"Error opening socket");
exit(-1);
}
setsockopt(create_socket,SOL_SOCKET,SO_REUSEADDR,(void*)&one,sizeof(one));
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons(5402);
fprintf(stderr,"Binding socket...");
while(bind(create_socket,(struct sockaddr *)&address,sizeof(address))!=0) {
fprintf(stderr,".");
usleep(1000000);
}
fprintf(stderr,"done\n");
listen(create_socket,3);
addrlen = sizeof(struct sockaddr_in);
// end sock code
binary = 1;
while ((c = getopt (argc, argv, "bd:o:q:s:V")) != EOF) {
switch (c) {
case 'b': /* PPM as binary file */
binary = 1;
break;
case 'd': /* change default device */
device = optarg;
break;
case 'o':
file = optarg;
break;
case 'q':
sscanf (optarg, "%d", &quality);
break;
case 's':
sscanf (optarg, "%dx%d", &width, &height);
break;
case 'V':
printf ("Vidcat, Version %s\n", VERSION);
exit (0);
break;
default:
usage (argv[0]);
break;
}
}
// if (file) {
/* out = fopen ("picture.ppm", "wb");
if (!out) {
perror ("picture.ppm");
return 1;
} */
// }
loop=1;
again:
new_socket = accept(create_socket,(struct sockaddr *)&address,&addrlen);
fprintf(stderr,"accept returned %d\n",new_socket);
if (new_socket<0) fprintf(stderr,"Error client connect\n");
fprintf(stderr,"client connected\n");
/* open the video4linux device */
while (max_try) {
dev = open (device, O_RDWR);
if (dev == -1) {
if (!--max_try) {
fprintf (stderr, "Can't open device %s\n", device);
close(create_socket);
return (1);
}
sleep (1);
} else { break; }
}
fprintf(stderr,"opened v4l device\n");
if (!num) {
/* if we loop we have to do this only once. so
* check frame number and execute only for the
* frame number "0".
*/
if (v4l_set_input (dev, input, norm) == -1) {
return (1);
}
if (v4l_check_size (dev, &width, &height) == -1) {
return (1);
}
/*if (v4l_check_palette (dev, &palette) == -1) {
return (1);
}*/
}
image = get_image (dev, width, height, palette, &size);
fprintf(stderr,"got image\n");
if (!size)
close (dev);
if (image) {
put_image_ppm (new_socket, image, width, height, binary);
fprintf(stderr,"put image\n");
//fclose (out);
close(new_socket);
fprintf(stderr,"closed socket\n");
if (size) {
munmap (image, size);
close (dev);
} else if (image) {
free (image);
}
if (loop) {
num=1;
goto again;
}
} else {
fprintf (stderr, "Error: Can't get image\n");
}
close(create_socket);
return (0);
}
#include <cassert>
#include "library.h"
#include "sign.h"
Library::Library()
{
}
Library::~Library()
{
}
bool Library::AddSign(Sign *newsign)
{ assert(newsign);
signs.push_back(newsign);
return true;
}
Sign * Library::GetSign(int room_number)
{ list<Sign*>::iterator i = signs.begin();
while(i != signs.end())
{ if(static_cast<Sign*>(*i)->GetRoomNumber() == room_number)
return static_cast<Sign*>(*i);
i++;
}
return NULL;
}
void Library::Print(FILE *fp)
{ list<Sign*>::iterator i = signs.begin();
while(i != signs.end())
{ static_cast<Sign*>(*i)->Print(fp);
i++;
}
}
#ifndef LIBRARY_H
#define LIBRARY_H
#include <list>
#include <cstdio>
#include "sign.h"
using namespace std;
class Library
{
public:
Library();
~Library();
Sign * GetSign(int room_number);
bool AddSign(Sign *newsign);
void Print(FILE *fp);
private:
list<Sign*> signs;
};
#endif
#include "sign.h"
Sign::Sign(Image *img, int rn, int x, int y)
{ image = img;
roomnum = rn;
locx = x;
locy = y;
}
Sign::~Sign()
{
}
int Sign::GetRoomNumber()
{ return roomnum;
}
Image * Sign::GetImage()
{ return image;
}
int Sign::GetLocationX()
{ return locx;
}
int Sign::GetLocationY()
{ return locy;
}
void Sign::Print(FILE *fp)
{ fprintf(fp,"Room %d \t%d, %d\n",roomnum,locx,locy);
}
#ifndef SIGN_H
#define SIGN_H
#include <cstdio>
class Image;
class Sign
{
public:
Sign(Image * img, int rn, int x, int y);
~Sign();
int GetRoomNumber();
Image * GetImage();
int GetLocationX();
int GetLocationY();
void Print(FILE *fp);
private:
Image * image;
int roomnum;
int locx;
int locy;
};
#endif
#include "imagesegregate.h"
#include <iostream>
#include <cmath>
using namespace std;
ImageSegregate::ImageSegregate(void)
{
}
ImageSegregate::~ImageSegregate(void)
{
}
void ImageSegregate::segregate(int xbounds, int ybounds)
{
vertical_autocrop();
x_boundaries = project(xbounds,0);
/* This crap is just for testing only */
/*
list<int*>::iterator xi = x_boundaries.begin();
list<int*>::iterator yi = y_boundaries.begin();
while(xi != x_boundaries.end())
{
for(int y = 0; y < height; y++)
{ data(**xi,y,RED) = 255;
data(**xi,y,GREEN) = 0;
data(**xi,y,BLUE) = 0;
}
xi++;
}
while(yi != y_boundaries.end())
{
for(int x = 0; x < width; x++)
{ data(x,**yi,RED) = 255;
data(x,**yi,GREEN) = 0;
data(x,**yi,BLUE) = 0;
}
yi++;
}*/
}
void ImageSegregate::vertical_autocrop()
{
int black = 0;
int oldy, y, x;
//Find the top and bottom edges of the black numbers
for(y = 0; y < height; y++)
{ for(x = 0; x < width; x++)
{ black += ( data(x,y,RED)+data(x,y,GREEN)+data(x,y,BLUE) <= 8 ? 1:0);
}
if(black > 0)
{ int *top = new int;
*top = y;
y_boundaries.push_back(top);
oldy = y;
break;
}
}
for(y = height; y > oldy; y--)
{ for(x = 0; x < width; x++)
{ black += ( data(x,y,RED)+data(x,y,GREEN)+data(x,y,BLUE) <= 8 ? 1:0);
}
if(black > 0)
{ int *bottom = new int;
*bottom = y;
y_boundaries.push_back(bottom);
break;
}
}
}
list<int*> ImageSegregate::project(int digits, int direction)
{
int param1, param2;
double threshold = 1.0;
double* intensity_plot;
list<int*> boundary;
if(direction == 0)
{ param1 = width;
param2 = height;
}
else
{ param1 = height;
param2 = width;
}
intensity_plot = new double[param1];
/* Get a plot of intensity values for each column of the image */
/*for(int x = 0; x < param1; x++)
{ intensity_plot[x] = 0;
for(int y = 1; y < param2-1; y++)
{ intensity_plot[x] += (data(x,y,RED) + data(x,y,GREEN) + data(x,y,BLUE)) / 3;
}
intensity_plot[x] /= param2 * maxval;
}*/
/*for(int x = 0; x < param1; x++)
{ intensity_plot[x] = 0;
for(int y = 0; y < param2; y++)
{ intensity_plot[x] += max(max(data(x,y,RED),data(x,y,GREEN)),data(x,y,BLUE));
}
intensity_plot[x] /= param2 * maxval;
}*/
/* Get a plot of the amount of whitish pixels in each band (row or column) */
for(int x = 0; x < param1; x++)
{ intensity_plot[x] = 0;
for(int y = 0; y < param2; y++)
{ intensity_plot[x] += (data(x,y,RED)>128||data(x,y,GREEN)>128||data(x,y,BLUE)>128?0:1);
}
}
int band = 1;
//Added 5/8/02 - Outer while loop for automatic
//threshold discovery
while(boundary.size() != digits && threshold > 0.0)
{
threshold -= 0.001;
band = 1;
boundary.clear();
while(band < param1)
{
/* Find a point that crosses the threshold */
//if((intensity_plot[band-1]<threshold && intensity_plot[band]>=threshold) ||
// (intensity_plot[band-1]>threshold && intensity_plot[band]<=threshold))
if((intensity_plot[band-1]<1 && intensity_plot[band]>=3) ||
(intensity_plot[band-1]>=3 && intensity_plot[band]<1))
{
/* Add boundary to the list */
int *b = new int;
*b = band;
boundary.push_back(b);
}
band++;
}
}
delete intensity_plot;
return boundary;
}
#ifndef IMAGESEGREGATE_H
#define IMAGESEGREGATE_H
#include <list>
#include "image.h"
using namespace std;
class ImageSegregate: public Image
{
public:
ImageSegregate(void);
~ImageSegregate(void);
void segregate(int xbounds, int ybounds);
list<int*> x_boundaries;
list<int*> y_boundaries;
private:
list<int*> project(int digits, int direction);
void vertical_autocrop();
};
#endif
#include <list>
#include "imagesegregate.h"
using namespace std;
int main(int argc, char **argv)
{
FILE *fin = fopen(argv[1],"r");
//FILE *fout = fopen("result.ppm","w");
ImageSegregate img;
img.input(fin);
img.segregate(4,2);
//img.output(fout);
list<int*>::iterator x = img.x_boundaries.begin();
list<int*>::iterator y = img.y_boundaries.begin();
list<int*>::iterator y2 = img.y_boundaries.end();
y2--;
while(x != img.x_boundaries.end())
{ printf("%d %d ",**x,**x);
printf("%d %d\n",**y,**y2);
x++;
}
fclose(fin);
//fclose(fout);
return 0;
}
#include "FindNumber.h"
FindNumber::FindNumber( void ) {
char* filename;
for ( int i = 0; i < 10; i++ ) {
switch( i ) {
case 0: filename = "zero.ppm"; break;
case 1: filename = "one.ppm"; break;
case 2: filename = "two.ppm"; break;
case 3: filename = "three.ppm"; break;
case 4: filename = "four.ppm"; break;
case 5: filename = "five.ppm"; break;
case 6: filename = "six.ppm"; break;
case 7: filename = "seven.ppm"; break;
case 8: filename = "eight.ppm"; break;
case 9: filename = "nine.ppm"; break;
}
FILE *fin = fopen( filename, "r" );
Image *img = new Image;
img->input( fin );
ideal_numbers[i] = img;
}
}
FindNumber::~FindNumber( void ) {
}
//old function definition
//void findNumber( Image number, double threshold ) {
void FindNumber::findNumber( Image* number ) {
double ratios[10];
int similar, num;
double count;
//const list<int*>& x = *numbers.x_boundaries;
//const list<int*>& y = *numbers.y_boundaries;
for ( int i = 0; i < 10; i++ ) {
/*
int left, right, bottom, top;
for ( list<int*>::iterator xi = x.begin(); xi != xi.end(); xi++ ) {
left = *xi;
xi++;
right = *xi;
list<int*>::iterator yi = y.begin();
bottom = *yi;
yi++;
top = *yi;
}
*/
printf( "%d\n", i );
Image* img = ideal_numbers[i];
if ( img->height != number->height || img->width != number->width ) {
printf( "Only do this once\n" );
number->scale( img->width, img->height );
}
similar = 0;
count = 0;
for ( int j = 0; j < img->width; j++ ) {
for ( int k = 0; k < img->height; k++ ) {
if ( number->data( j, k, RED ) == img->data( j, k, RED ) )
similar++;
if ( number->data( j, k, BLUE ) == img->data( j, k, BLUE ) )
similar++;
if ( number->data( j, k, GREEN ) == img->data( j, k, GREEN ) )
similar++;
count += 3.0;
}
}
ratios[i] = similar / count;
}
num = -1;
double maxval = -1;
for ( int i = 0; i < 10; i++ ) {
if ( ratios[i] > maxval ) {
maxval = ratios[i];
num = i;
}
/*
if ( ratios[i] > threshold ) {
if ( num == -1 )
num = i;
else
if ( ratios[i] > ratios[num] )
num = i;
}
*/
}
printf( "%d %f\n", num, maxval );
}
#ifndef FINDNUMBER_H
#define FINDNUMBER_H
#include "image.h"
class FindNumber {
public:
FindNumber( void );
~FindNumber( void );
//void findNumber( Image numbers, double threshold );
void findNumber( Image* number );
private:
Image* ideal_numbers[10];
};
#endif
#include <list>
#include "FindNumber.h"
using namespace std;
int main(int argc, char **argv)
{
//Open the file given on then command line and make
//sure its open.
FILE *fin = fopen(argv[1],"r");
if(fin == NULL)
{ printf("Your file sucks. What's wrong with you?\n");
return -1;
}
Image *img = new Image;
FindNumber fn;
//fn = new FindNumber();
img->input( fin );
fn.findNumber( img );
fclose(fin);
return 0;
}
<?php
$host = '';
$port = 0;
session_start();
//First invocation of script so initialize the session
if(!session_is_registered("is_connected"))
{ session_register("is_connected");
session_register("host");
session_register("port");
$is_connected = false;
}
//If we're not connected then respond to connect button clicks
if(!$is_connected)
{ if(isset($connect_button))
{ $host = $host_text;
$port = $port_text;
if(!Connect())
{ exit("Error: Unable to connect socket");
}
else
{ $is_connected = true;
}
}
}
//If we are connected then respond to robot control buttons
else if($is_connected)
{ if(isset($forward_button))
{ Send("setVel 100");
}
if(isset($reverse_button))
{ Send("setVel -100");
}
if(isset($left_button))
{ Send("turn -10");
}
if(isset($right_button))
{ Send("turn 10");
}
if(isset($stop_button))
{ Send("stop");
}
if(isset($hidag_button))
{ Send("say Hi Dag");
}
if(isset($refresh_button))
{ //TODO: call some function to refresh the image from the cam
}
}
else
{ //Don't know how we could get here, should probably change the previous else if to just an else.
}
//Test the connection, also creates a persistent connection so that subsequent opens and closes
//do not incurr additional overhead (as I understand it).
//CHANGED ??/??/02 - robot controller disallows persistent connections (pfsockopen to fsockopen)
function Connect()
{ global $host, $port;
$socket = fsockopen($host,$port);
if(!$socket) return false;
fclose($socket);
return true;
}
//Send a command over the socket
function Send($command)
{ global $host, $port;
$socket = fsockopen($host, $port);
if(!$socket) return false;
if(fputs($socket,$command) != strlen($command))
{ echo "Error: Unable to send command: $command";
return false;
}
fclose($socket);
return true;
}
?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD><TITLE>Robot GUI</TITLE>
<META content=" name=GENERATOR">
<meta name="vs_targetSchema" content="http://schemas.microsoft.com/intellisense/ie5">
</HEAD>
<body>
<P align="center"><?php echo "Connection status: $is_connected";?></P>
<form action="robotgui.php" method="post">
<DIV align="center">
<DIV style="DISPLAY: inline; WIDTH: 40px; HEIGHT: 19px" ms_positioning="FlowLayout">Host:</DIV>
<INPUT id="host_text" type="text" name="host_text">
<DIV style="DISPLAY: inline; WIDTH: 40px; HEIGHT: 19px" ms_positioning="FlowLayout">Port:</DIV>
<INPUT id="port_text" style="WIDTH: 83px; HEIGHT: 22px" type="text" size="8" name="port_text"></DIV>
<DIV align="center">
<INPUT id="connect_button" type="submit" value="Connect" name="connect_button"></DIV>
<P align="center"><IMG height="240" alt="image from webcam" src="" width="320" align="absMiddle"></P>
<P align="center">
<INPUT id="stop_button" type="submit" value="Stop" name="stop_button">
<INPUT id="forward_button" type="submit" value="Forward" name="forward_button">
<INPUT id="reverse_button" type="submit" value="Reverse" name="reverse_button">
<INPUT id="left_button" type="submit" value="Left" name="left_button">
<INPUT id="right_button" type="submit" value="Right" name="right_button">
</P>
<P align="center">
<INPUT id="hidag_button" type="submit" value="Hi DAG" name="hidag_button">
<INPUT id="refresh_button" type="submit" value="Refresh" name="refresh_button">
</P>
</form>
</body>
</HTML>
#include <stdio.h>
#include <string.h>
#include <math.h>
#include <malloc.h>
#include "image.h"
#include "llist.h"
#define RED 0
#define BLUE 1
#define GREEN 2
#define PI 3.14159
#define sqr(x) ((x)*(x))
#define pair(x,y) (pairs[(x)*numlines + (y)])
// Constructors
Image::Image() {
width = height = 0;
maxval = 1; pixeldata = NULL;
}
// Initialize a raster of size h by w pixels
// with maximum pixel color data maxv
Image::Image(int h, int w, int maxv) {
fflush(stdout);
width = w; height = h;
maxval = maxv;
pixeldata = new int[3*w*h];
}
// Destructor
Image::~Image() {
if(width)
delete pixeldata;
}
// Read in any file PNM, PPM, or PGM file
// (consult the PNM manual page)
void
Image::input(FILE *input) {
/* This method will read in an image
* in the PNM file format, specifically
* the PPM raw (P6) format.
*/
unsigned char ch1, ch2;
int x, y, type, r, g, b;
// PNM Header
fscanf(input, "P%d", &type);
if(type == 2) {
// PGM plain
fscanf(input, "%d%d%d", &width, &height, &maxval);
fprintf(stderr, "Input type PGM plain (P2), %d x %d pixels, color depth %d\n", width, height, maxval);
pixeldata = new int[3*width*height];
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
fscanf(input, "%d", &r);
data(x,y,RED) = r;
data(x,y,GREEN) = r;
data(x,y,BLUE) = r;
}
}
}
else if(type == 3) {
// PPM plain
fscanf(input, "%d%d%d", &width, &height, &maxval);
fprintf(stderr, "Input type PPM plain (P3), %d x %d pixels, color depth %d\n", width, height, maxval);
pixeldata = new int[3*width*height];
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
fscanf(input, "%d%d%d", &r, &g, &b);
data(x,y,RED) = r;
data(x,y,GREEN) = g;
data(x,y,BLUE) = b;
}
}
}
else if(type == 5) {
// PGM RAW
fscanf(input, "%d%d%d", &width, &height, &maxval);
fprintf(stderr, "Input type PGM raw (P5), %d x %d pixels, color depth %d\n", width, height, maxval);
fscanf(input, "%*c");
pixeldata = new int[3*width*height];
// use one or two bytes
if(maxval < 256) {
for(y = 0; y<height; y++) {
for(x=0; x<width; x++) {
fread(&ch1, 1, 1, input);
data(x,y,RED) = ch1;
data(x,y,GREEN) = ch1;
data(x,y,BLUE) = ch1;
}
}
}
else {
for(y = 0; y<height; y++) {
for(x=0; x<width; x++) {
fread(&ch1, 1, 1, input);
fread(&ch2, 1, 1, input);
data(x,y,RED) = 256*ch1+ch2;
data(x,y,GREEN) = 256*ch1+ch2;
data(x,y,BLUE) = 256*ch1+ch2;
}
}
}
}
else if(type == 6) {
// PPM RAW
fscanf(input, "%d%d%d", &width, &height, &maxval);
fprintf(stderr, "Input type PPM raw (P6), %d x %d pixels, color depth %d\n", width, height, maxval);
fscanf(input, "%*c");
pixeldata = new int[3*width*height];
// use one or two bytes
if(maxval < 256) {
for(y = 0; y<height; y++) {
for(x=0; x<width; x++) {
fread(&ch1, 1, 1, input);
data(x,y,RED) = ch1;
fread(&ch1, 1, 1, input);
data(x,y,GREEN) = ch1;
fread(&ch1, 1, 1, input);
data(x,y,BLUE) = ch1;
}
}
}
else {
for(y = 0; y<height; y++) {
for(x=0; x<width; x++) {
fread(&ch1, 1, 1, input);
fread(&ch2, 1, 1, input);
data(x,y,RED) = 256*ch1+ch2;
fread(&ch1, 1, 1, input);
fread(&ch2, 1, 1, input);
data(x,y,GREEN) = 256*ch1+ch2;
fread(&ch1, 1, 1, input);
fread(&ch2, 1, 1, input);
data(x,y,BLUE) = 256*ch1+ch2;
}
}
}
}
else {
fprintf(stderr, "Error: file type not recognized.\n");
fflush(stdout);
}
}
// make the current image a grayscale image
void
Image::grayscale() {
int x, y, grey;
double R, G, B;
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
R = data(x,y,RED);
G = data(x,y,GREEN);
B = data(x,y,BLUE);
grey = (int)(0.299*R + 0.587*G + 0.114*B);
data(x,y,RED) = data(x,y,GREEN) = data(x,y,BLUE) = grey;
}
}
}
// Make the current image a binary image
// with luminescence threshold 0.7 (70%)
void
Image::binarize() {
binarize(0.7);
}
// with arbitrary threshold
void
Image::binarize(double threshold) {
binarize((int)((double)maxval * threshold));
}
// or pixel brightness threshold
void
Image::binarize(int threshold) {
int x, y, grey;
double R, G, B;
maxval = 1;
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
R = data(x,y,RED);
G = data(x,y,GREEN);
B = data(x,y,BLUE);
grey = (int)(0.299*R + 0.587*G + 0.114*B);
if(grey > threshold)
data(x,y,RED) = data(x,y,GREEN) = data(x,y,BLUE) = 1;
else
data(x,y,RED) = data(x,y,GREEN) = data(x,y,BLUE) = 0;
}
}
}
// equalize an image
void
Image::equalize() {
int *counts = new int[maxval];
int *mapsto = new int[maxval];
int x, y, grey, grey2;
double R, G, B;
for(grey=0; grey<maxval; grey++) {
counts[grey] = 0;
}
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
R = data(x,y,RED);
G = data(x,y,GREEN);
B = data(x,y,BLUE);
grey = (int)(0.299*R + 0.587*G + 0.114*B);
counts[grey]++;
}
}
for(grey=0; grey<maxval; grey++) {
mapsto[grey] = 0;
for(grey2 = 0; grey2<grey; grey2++) {
mapsto[grey] += counts[grey2];
}
mapsto[grey] *= maxval;
mapsto[grey] /= (width*height);
}
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
R = data(x,y,RED);
G = data(x,y,GREEN);
B = data(x,y,BLUE);
grey = (int)(0.299*R + 0.587*G + 0.114*B);
grey = mapsto[grey];
data(x,y,RED) = data(x,y,GREEN) = data(x,y,BLUE) = grey;
}
}
}
// output the current image (PNM, consult the manual
// page again)
void
Image::output(FILE *out, char* fName) {
char* extension = rindex(fName, '.');
if(extension == NULL) return output(out);
if(!strcasecmp(extension, ".pgm"))
return outputPGMraw(out);
if(!strcasecmp(extension, ".ppm"))
return outputPPMraw(out);
if(!strcasecmp(extension, ".dat"))
return outputDAT(out);
return output(out);
}
// if no type is specified, output as a
// PPM
void
Image::output(FILE *output) {
outputPPM(output);
}
// output the current image as a plain PGM
void
Image::outputPGM(FILE *output) {
/* writes out the file, but in
* plain PGM format (P2)
*/
int x, y;
fprintf(stderr, "Output type PGM plain (P2), %d x %d pixels, color depth %d\n", width, height, maxval);
fprintf(output, "P2\n%d %d\n%d\n", width, height, maxval);
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
fprintf(output, "%d\n", data(x,y,RED));
}
}
}
// output the current file as a plain PPM
void
Image::outputPPM(FILE *output) {
/* writes out the file, but in
* plain PPM format (P3)
*/
int x, y;
fprintf(stderr, "Output type PPM plain (P3), %d x %d pixels, color depth %d\n", width, height, maxval);
fprintf(output, "P3\n%d %d\n%d\n", width, height, maxval);
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
fprintf(output, "%d %d %d\n", data(x,y,RED), data(x,y,GREEN), data(x,y,BLUE));
}
}
}
// output the current image as a raw PGM
void
Image::outputPGMraw(FILE *output) {
/* writes out the file, but in
* raw PPM format (P6)
*/
int x, y;
char ch1, ch2;
fprintf(stderr, "Output type PGM raw (P5), %d x %d pixels, color depth %d\n", width, height, maxval);
fprintf(output, "P5\n%d %d\n%d\n", width, height, maxval);
if(maxval < 256) {
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
ch1 = data(x,y,RED);
fwrite(&ch1, 1, 1, output);
}
}
}
else {
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
ch1 = data(x,y,RED)/256;
ch2 = data(x,y,RED)%256;
fwrite(&ch1, 1, 1, output);
fwrite(&ch2, 1, 1, output);
}
}
}
}
// output the image as a raw PPM
void
Image::outputPPMraw(FILE *output) {
/* writes out the file, but in
* raw PPM format (P6)
*/
int x, y;
char ch1, ch2;
fprintf(stderr, "Output type PPM raw (P6), %d x %d pixels, color depth %d\n", width, height, maxval);
fprintf(output, "P6\n%d %d\n%d\n", width, height, maxval);
if(maxval < 256) {
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
ch1 = data(x,y,RED);
fwrite(&ch1, 1, 1, output);
ch1 = data(x,y,GREEN);
fwrite(&ch1, 1, 1, output);
ch1 = data(x,y,BLUE);
fwrite(&ch1, 1, 1, output);
}
}
}
else {
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
ch1 = data(x,y,RED)/256;
ch2 = data(x,y,RED)%256;
fwrite(&ch1, 1, 1, output);
fwrite(&ch2, 1, 1, output);
ch1 = data(x,y,GREEN)/256;
ch2 = data(x,y,GREEN)%256;
fwrite(&ch1, 1, 1, output);
fwrite(&ch2, 1, 1, output);
ch1 = data(x,y,BLUE)/256;
ch2 = data(x,y,BLUE)%256;
fwrite(&ch1, 1, 1, output);
fwrite(&ch2, 1, 1, output);
}
}
}
}
void
Image::outputDAT(FILE* output) {
/* writes out the file as
* ASCII data
*/
int x, y;
fprintf(stderr, "Output type plain ascii, %d x %d pixels, color depth %d\n", width, height, maxval);
// fprintf(output, "P2\n%d %d\n%d\n", width, height, maxval);
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
fprintf(output, "%d %d %d\n", x, y, data(x,y,RED));
}
fprintf(output, "\n");
}
}
// edge detect an image using the canny
// edge detector.
Image*
Image::edge(void) {
int *row0, *row1, *row2, *tmprow;
int i, row;
double sum1, sum2, sum;
Image *temp;
temp = new Image(height, width, maxval);
row0 = new int[width];
row1 = new int[width];
row2 = new int[width];
tmprow = new int[width];
for(i=0; i<width; i++)
temp->data(i, 0, RED) =
temp->data(i, 0, GREEN) =
temp->data(i, 0, BLUE) =
temp->data(i, width-1, RED) =
temp->data(i, width-1, GREEN) =
temp->data(i, width-1, BLUE) = 0;
for(i=0; i<width; i++) {
row0[i] = data(i, 0, RED);
row1[i] = data(i, 1, RED);
}
for(row = 1; row < height - 1; row++) {
for(i=0; i<width; i++)
row2[i] = data(i, row+1, RED);
temp->data(0, row, RED) =
temp->data(0, row, GREEN) =
temp->data(0, row, BLUE) =
temp->data(width-1, row, RED) =
temp->data(width-1, row, GREEN) =
temp->data(width-1, row, BLUE) = 0;
for(i=1; i<width-1; i++) {
sum1 = row0[i+1] - row0[i-1] + 2*(row1[i+1] - row1[i-1]) + row2[i+1] - row2[i-1];
sum2 = row2[i-1] + 2*row2[i] + row2[i+1] - row0[i-1] - 2*row0[i] - row0[i+1];
sum = sqrt(sum1*sum1 + sum2*sum2);
sum /= 1.8;
if(sum > maxval) sum = maxval;
temp->data(i, row, RED) =
temp->data(i, row, GREEN) =
temp->data(i, row, BLUE) = (int)sum;
}
tmprow = row0;
row0 = row1;
row1 = row2;
row2 = tmprow;
}
return temp;
}
// crop an image
void
Image::crop(int minx, int maxx, int miny, int maxy) {
int *newdata, x, y, maxv = 0;
minx = (minx>=0)?minx:0;
miny = (miny>=0)?miny:0;
maxx = (maxx<width)?maxx:width-1;
maxy = (maxy<height)?maxy:height-1;
newdata = new int[(maxx-minx+1)*(maxy-miny+1)*3];
for(x = minx; x <= maxx; x++) {
for(y = miny; y <= maxy; y++) {
newdata[(x-minx)*(maxy-miny+1)*3 + (y-miny)*3 + RED] = data(x,y,RED);
newdata[(x-minx)*(maxy-miny+1)*3 + (y-miny)*3 + GREEN] = data(x,y,GREEN);
newdata[(x-minx)*(maxy-miny+1)*3 + (y-miny)*3 + BLUE] = data(x,y,BLUE);
maxv = (maxv > data(x,y,RED))?maxv:data(x,y,RED);
maxv = (maxv > data(x,y,BLUE))?maxv:data(x,y,BLUE);
maxv = (maxv > data(x,y,GREEN))?maxv:data(x,y,GREEN);
}
}
delete pixeldata;
pixeldata = newdata;
height = maxy - miny + 1;
width = maxx - minx + 1;
maxval = maxv;
}
// scale an image
void
Image::scale(int newwidth, int newheight) {
int *newdata, x, y;
float xs, ys;
newwidth = (newwidth>=1)?newwidth:1;
newheight = (newheight>=1)?newheight:1;
xs = ((float)newwidth)/((float)width);
ys = ((float)newheight)/((float)height);
newdata = new int[(newwidth)*(newheight)*3];
for(x = 0; x < newwidth; x++) {
for(y = 0; y < newheight; y++) {
newdata[x*newheight*3+y*3+RED]=data((int)((float)x/xs),(int)((float)y/ys),RED);
newdata[x*newheight*3+y*3+BLUE]=data((int)((float)x/xs),(int)((float)y/ys),BLUE);
newdata[x*newheight*3+y*3+GREEN]=data((int)((float)x/xs),(int)((float)y/ys),GREEN);
}
}
delete pixeldata;
pixeldata = newdata;
height = newheight;
width = newwidth;
}
// find the inside border of a sign that has
// been found and find the corners
void
Image::findborder() {
int x0=0, y0=height/2, x, y, xs, ys, i, j;
int numold, num, dx, dy;
int haschanged, maxvs[4];
int nogood;
int counts[180000], xi, yi;
int theta; double r;
int maxrs[4], maxts[4], ri, k;
double r1, r2, th1, th2;
int xpoints[4], ypoints[4];
while(data(x0,y0,RED) != 0 && x0 < width) x0++;
while(data(x0,y0,RED) == 0 && x0 < width) x0++;
// we've just passed the border
for(x=0; x<width; x++)
for(y=0; y<height; y++)
data(x,y,BLUE) = data(x,y,GREEN) = 0;
data(x0,y0,BLUE) = 1;
numold = 0; num = 1;
while(numold != num) {
numold = num;
for(x=0; x<width; x++)
for(y=0; y<height; y++) {
if(!data(x,y,BLUE)) continue;
for(dx=-1; dx<=1; dx++) {
for(dy=-1; dy<=1; dy++) {
if(data(x+dx,y+dy,RED) && !data(x+dx,y+dy,BLUE) &&
!(data(x+dx-1,y+dy,RED) && data(x+dx+1,y+dy,RED)
&& data(x+dx,y+dy-1,RED) && data(x+dx,y+dy+1,RED)
&& data(x+dx-1,y+dy-1,RED) && data(x+dx-1,y+dy+1,RED)
&& data(x+dx+1,y+dy-1,RED)
&& data(x+dx+1,y+dy+1,RED))) {
data(x+dx,y+dy,GREEN) = 1; num++;
}
}
}
}
for(x=0; x<width; x++)
for(y=0; y<height; y++) {
data(x,y,BLUE) |= data(x,y,GREEN);
data(x,y,GREEN) = 0;
}
}
// found the border, find the corners
for(theta = 0; theta < 360; theta++)
for(ri=0; ri<500; ri++)
counts[ri*360+theta] = 0;
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
if(data(x,y,BLUE)) {
for(theta=0; theta<360; theta++) {
r = (double)(x) * cos(PI/180.0 * (double)theta)
+ (double)(y)*sin(PI/180.0 * (double)(theta));
if(0 <= r && r < 500) {
counts[(int)(r)*360+ theta]++;
}
}
}
}
}
for(i=0; i<4; i++) {
maxvs[i] = maxrs[i] = maxts[i] = 0;
for(theta=0; theta<360; theta++) {
for(ri=0; ri<500; ri++) {
nogood = 0;
for(j=0; j<i; j++) {
if(( ((ri - maxrs[j] +500)%500 <= 10) || (maxrs[j] - ri + 500)%500 <= 10)
|| (((theta - maxts[j]+360)%360 <= 10) && ((maxts[j] - theta + 360)%360 <= 10))) {
nogood = 1; // too close
}
}
if(!nogood && counts[ri*360 + theta] > maxvs[i]) {
maxrs[i] = ri;
maxts[i] = theta;
maxvs[i] = counts[ri*360 + theta];
}
}
}
}
for(i=0; i<4; i++) {
fprintf(stderr, "r = %d, th = %d, counts = %d\n", maxrs[i], maxts[i], maxvs[i]);
}
k=0;
for(i=0; i<4; i++) {
for(j=0; j<4; j++) {
r1 = (double) maxrs[i];
r2 = (double) maxrs[j];
th1 = PI/180.0 * (double) maxts[i];
th2 = PI/180.0 * (double) maxts[j];
xi = (int)( (r2*sin(th1) - r1*sin(th2))/(sin(th1 - th2)));
yi = (int)( (r1*cos(th2) - r2*cos(th1))/(sin(th1 - th2)));
// printf("(%d, %d)\n", xi, yi);
if(xi >= 0 && xi < width && yi >= 0 && yi < height) {
xpoints[k] = xi; ypoints[k] = yi;
k++;
data(xi, yi, RED) = 1;
data(xi, yi, BLUE) = 1;
data(xi, yi, GREEN) = 1;
}
}
}
for(x=0; x<width; x++) {
for(y=0; y<height; y++) {
if(!data(x,y,BLUE))
data(x,y,BLUE) = data(x,y,GREEN) = data(x,y,RED);
else if(!data(x,y,GREEN)) {
data(x,y,RED) = 0;
data(x,y,GREEN) = 1;
data(x,y,BLUE) =0;
}
}
}
}
/* Hough Transform */
// hough transform an image, find maxima in the
// transform, find the limits of the line, and
// find all possible quadrilaterals with interior
// angle larger than 60 degrees, and output the
// one with the largest area. This is what we
// think the sign is.
Image*
Image::hough(double threshold, int bs, int mincounts) {
int rres = 560, thresh;
int x, y, xc, yc, xm, ym, yin, i, theta, max;
double r, m, b;
Image *temp;
LList *current, *current2, *current3, *current4;
int xi, yi, i1, i2, i3, i4, j;
double r1, r2, th1, th2;
int intersect1, intersect2, bs2 = 15;
DLList **pairs;
int numlines;
// binarize(0.10);
temp = new Image(360, rres, maxval);
for(y=0; y<360; y++)
for(x=0; x<rres; x++)
(*temp).data(x,y,RED) = 0;
// this is the hough transform itself
for(y=0; y<height; y++) {
for(x=0; x<width; x++) {
if(data(x,y,RED)) {
for(theta=0; theta<360; theta++) {
r = (double)(x) * cos(PI/180.0 * (double)theta)
+ (double)(y)*sin(PI/180.0 * (double)(theta));
if(0 <= r && r/2 <= rres) {
(*temp).data((int)(r/2), theta, RED)++;
}
}
}
}
}
// this loop just sets the image parameters
// (maximum brightness, and the other colors)
// so that it outputs correctly if that's what
// the programmer wants
temp->maxval = 1;
for(y=0; y<360; y++) {
for(x=0; x<rres; x++) {
if(temp->maxval < (*temp).data(x,y,RED)) {
temp->maxval = (*temp).data(x,y,RED);
xm = x; ym = y;
}
(*temp).data(x,y,BLUE) = (*temp).data(x,y,GREEN) = (*temp).data(x,y,RED);
}
}
fprintf(stderr, "Maximum value at (%d, %d).\n", xm, ym);
head = tail = NULL;
headd = taild = NULL;
numlines = 0;
// find candidate lines
thresh = (int)(threshold*((double) temp->maxval));
for(yc=0; yc<360; yc++) {
for(xc=bs; xc<rres-bs; xc++) {
if((*temp).data(xc,yc,RED) < thresh ||
(*temp).data(xc,yc,RED) < mincounts) continue;
max = 1;
for(y=yc-bs; y<yc+bs; y++) {
for(x=xc-bs; x<xc+bs; x++) {
yin = (y + 360)%360;
if((*temp).data(x,yin,RED) >= (*temp).data(xc,yc,RED)
&& (x != xc || yin != yc))
max = 0;
}
}
// max=1;
if(max) {
// this means it was a line
fprintf(stderr, "Maxima at (%d, %d).\n", xc, yc);
// find the endpoints of the line and
// color the line on the image
int xi=0, xf=0, xs=-1, misd=0;
if((yc > 45 && yc < 135) || (yc > 225 && yc < 315)) {
m = -1/(tan(PI/180.0 * (double)yc));
b = ((double)(2*xc))/(sin(PI/180.0 * (double)yc));
fprintf(stderr, "m = %lf, b = %lf\n", m, b);
for(x=0; x<width; x++) {
y=(int)(m*(double)x + b);
if(0 <= y && y < height) {
if(data(x,y,RED) || data(x,y-1,RED) || data(x,y+1,RED)
|| data(x,y-2,RED) || data(x,y+2,RED)
|| data(x,y-3,RED) || data(x,y+3,RED)) {
misd = 0;
if(xs == -1)
xs = x;
}
else if(xs != -1) {
if(++misd > 3) {
if(x - misd - xs > xf - xi) {
xi = xs; xf = x - misd;
}
xs=-1;
misd=0;
}
}
}
if(xs!=-1) {
if(x - 1 - xs > xf - xi) {
xi = xs; xf = x - 1;
}
}
// data(x,y,RED)=1; data(x,y,GREEN)=0; data(x,y,BLUE)=1;
}
for(x=xi; x<xf; x++) {
y=(int)(m*(double)x + b);
data(x,y,RED)=0;
data(x,y,GREEN)=1;
data(x,y,BLUE)=0;
}
if(xf - xi > 0) {
appendline((double)(2*xc), (double)yc, xi, xf, (int)(m*(double)xi+b), (int)(m*(double)xf+b));
numlines++;
}
}
else {
m = -(tan(PI/180.0 * (double)yc));
b = ((double)(2*xc))/(cos(PI/180.0 * (double)yc));
for(y=0; y<height; y++) {
x=(int)(m*(double)y + b);
if(0 <= x && x < width) {
if(data(x,y,RED) || data(x-1,y,RED) || data(x+1,y,RED)
|| data(x-2,y,RED) || data(x+2,y,RED)
|| data(x-3,y,RED) || data(x+3,y,RED)) {
misd = 0;
if(xs == -1)
xs = y;
}
else if(xs != -1) {
if(++misd > 3) {
if(y - misd - xs > xf - xi) {
xi = xs; xf = y - misd;
}
xs=-1;
misd=0;
}
}
}
if(xs!=-1) {
if(y - 1 - xs > xf - xi) {
xi = xs; xf = y - 1;
}
}
// data(x,y,GREEN)=1; data(x,y,BLUE)=0; data(x,y,RED)=1;
}
for(y=xi; y<xf; y++) {
x=(int)(m*(double)y + b);
data(x,y,RED)=0;
data(x,y,GREEN)=0;
data(x,y,BLUE)=1;
}
if(xf - xi > 0) {
appendline((double)(2*xc), (double)yc, (int)(m*(double)xi+b), (int)(m*(double)xf+b), xi, xf);
numlines++;
}
}
}
}
}
// find pairs of intersecting lines
pairs = (DLList**)malloc(sizeof(DLList*)*numlines*numlines);
for(i=0; i<numlines*numlines; i++)
pairs[i] = NULL;
for(current = head, i=0;
current != NULL;
current = current -> next, i++) {
for(current2 = head, j=0;
current2 != NULL;
current2 = current2 -> next, j++) {
if(current->theta - current2->theta > 60 || current2->theta - current->theta > 60) {
r1 = (double) current->r;
r2 = (double) current2->r;
th1 = PI/180.0 * (double) current->theta;
th2 = PI/180.0 * (double) current2->theta;
xi = (int)( (r2*sin(th1) - r1*sin(th2))/(sin(th1 - th2)));
yi = (int)( (r1*cos(th2) - r2*cos(th1))/(sin(th1 - th2)));
// one possible algorithm: find the intersection point
// of the *entire* line (ignoring the endpoints)
// and if it lands on a pixel that is "on" in the image,
// then we'll consider them as having intersected.
// this is primitive.
/*
printf("(%d, %d) for (%lf, %lf) and (%lf, %lf)\n", xi, yi, r1, th1, r2, th2);
if(xi >= 0 && xi < width && yi >= 0 && yi < height) {
// the intersection point is inside the picture
intersect1 = 0;
intersect2 = 0;
for(x=xi-bs2; x<xi + bs2; x++) {
if(x >= width || x < 0) continue;
for(y=yi-bs2; y<yi + bs2; y++) {
if(y >= height || y < 0) continue;
if(data(x,y,RED)) {
if(((cos(th1) != 0 && fabs((double)x - (r1 - (double)y*sin(th1))/(cos(th1))) < 1.0))
|| ((sin(th1) != 0 && fabs((double)y - (r1 - (double)x*cos(th1))/(sin(th1))) < 1.0)))
intersect1++;
if(((cos(th2) != 0 && fabs((double)x - (r2 - (double)y*sin(th2))/(cos(th2))) < 1.0))
|| ((sin(th2) != 0 && fabs((double)y - (r2 - (double)x*cos(th2))/(sin(th2))) < 1.0)))
intersect2++;
}
}
}
if(intersect1 > bs2/3 && intersect2 > bs2/3) {
pair(i, j) = pair(j, i) = appenddlines(r1, th1, r2, th2, xi, yi);
printf("Lines intersect.\n");
data(xi,yi,RED) = 1;
data(xi,yi,GREEN) = 0;
data(xi,yi,BLUE) = 0;
}
}
*/
// another possible algorithm -- if the endpoints are
// within 10 pixels of each other, then consider the
// lines as having intersected. this is also primitive
// but appropriate in the context of sign finding, because
// the edges (which are the lines we are trying to find)
// are supposed to intersect at the endpoints
/*
if((sqr(current->x1 - current2->x1) + sqr(current->y1 - current2->y1) < 100) ||
(sqr(current->x1 - current2->x2) + sqr(current->y1 - current2->y2) < 100) ||
(sqr(current->x2 - current2->x2) + sqr(current->y2 - current2->y2) < 100) ||
(sqr(current->x2 - current2->x1) + sqr(current->y2 - current2->y1) < 100)
){
*/
// this is the final algorithm. this algorithm
// simply combines the previous two, in a way.
// it finds the intersection point of the two
// lines and then considers whether or not it
// is "close" to the interior of the line (i.e.,
// whether or not it is actually on the line or
// if it is near the endpoints). This works *extremely*
// well.
if(((xi+10 >= current->x1 && xi-10 <= current->x2) || (xi+10 >= current->x2 && xi-10 <= current->x1 )) &&
((yi+10 >= current->y1 && yi-10 <= current->y2) || (yi+10 >= current->y2 && yi-10 <= current->y1 )) &&
((xi+10 >= current2->x1 && xi-10 <= current2->x2) || (xi+10 >= current2->x2 && xi-10 <= current2->x1 )) &&
((yi+10 >= current2->y1 && yi-10 <= current2->y2) || (yi+10 >= current2->y2 && yi-10 <= current2->y1 ))) {
pair(i, j) = pair(j, i) = appenddlines(r1, th1, r2, th2, xi, yi);
// printf("Lines intersect.\n");
data(xi,yi,RED) = 1;
data(xi,yi,GREEN) = 0;
data(xi,yi,BLUE) = 0;
}
}
}
}
// find all quadruplets of lines that satisfy
// the property that there are two pairs of lines
// such that each pair is "parallel" (i.e., do not
// intersect) but any other pair of lines from the
// quadruplet *do* intersect. If you think about
// this, this is the configuration of a quadrilateral.
// declare current3, current4
// declare ints i1, i2, i3, i4
int minx, maxx, miny, maxy;
int bestminx, bestmaxx, bestminy, bestmaxy;
int maxarea = 0;
for(current = head, i1 = 0; current != NULL; current = current->next, i1++) {
for(current2 = current->next, i2 = i1+1; current2 != NULL; current2 = current2->next, i2++) {
for(current3 = current->next, i3 = i1+1; current3 != NULL; current3 = current3->next, i3++) {
for(current4 = current3->next, i4 = i3+1; current4 != NULL; current4 = current4->next, i4++) {
if(pair(i1, i3) && pair(i1, i4) && pair(i2, i3) && pair(i2, i4) && !pair(i1, i2) && !pair(i3, i4)) {
fprintf(stderr, "We have a quadrilateral:\n");
//printf("(%f, %f) || (%f, %f) and (%f, %f) || (%f, %f)\n", current->r, current->theta, current2->r,
current2->theta, current3->r, current3->theta, current4->r, current4->theta);
fprintf(stderr, "(%d, %d), (%d, %d), (%d, %d), (%d, %d)\n", pair(i1,i3)->xi, pair(i1,i3)->yi,
pair(i1,i4)->xi, pair(i1,i4)->yi, pair(i2,i3)->xi, pair(i2,i3)->yi, pair(i2,i4)->xi, pair(i2,i4)->yi);
minx = maxx = pair(i1,i3)->xi;
miny = maxy = pair(i1,i3)->yi;
if(minx > pair(i1,i4)->xi) minx = pair(i1,i4)->xi;
if(minx > pair(i2,i3)->xi) minx = pair(i2,i3)->xi;
if(minx > pair(i2,i4)->xi) minx = pair(i2,i4)->xi;
if(maxx < pair(i1,i4)->xi) maxx = pair(i1,i4)->xi;
if(maxx < pair(i2,i3)->xi) maxx = pair(i2,i3)->xi;
if(maxx < pair(i2,i4)->xi) maxx = pair(i2,i4)->xi;
if(miny > pair(i1,i4)->yi) miny = pair(i1,i4)->yi;
if(miny > pair(i2,i3)->yi) miny = pair(i2,i3)->yi;
if(miny > pair(i2,i4)->yi) miny = pair(i2,i4)->yi;
if(maxy < pair(i1,i4)->yi) maxy = pair(i1,i4)->yi;
if(maxy < pair(i2,i3)->yi) maxy = pair(i2,i3)->yi;
if(maxy < pair(i2,i4)->yi) maxy = pair(i2,i4)->yi;
if(maxarea < (maxx-minx)*(maxy-miny)) {
bestminx = minx;
bestminy = miny;
bestmaxx = maxx;
bestmaxy = maxy;
maxarea = (maxx-minx)*(maxy-miny);
}
fprintf(stderr, "a %dx%d region\n", maxx-minx, maxy-miny);
}
}
}
}
}
// if we found any quadrilateral at all,
// output the one with the largest area
// and mark it with a box in the output
// image. This is the part of the image
// that will be cropped for further analysis.
if(maxarea > 0) {
// printf("%d %d %d %d\n", bestminx, bestmaxx, bestminy, bestmaxy);
printf("%d %d %d %d\n", bestminx-10, bestmaxx+10, bestminy-10, bestmaxy+10);
fprintf(stderr, "a %d region\n", maxarea);
fprintf(stderr, "%dx%d\n", bestmaxx-bestminx, bestmaxy-bestminy);
}
for(y=-1;y<=1;y++) {
for(x=bestminx; x<=bestmaxx; x++) {
data(x,bestminy+y,RED)=1;
data(x,bestminy+y,BLUE)=0;
data(x,bestminy+y,GREEN)=0;
data(x,bestmaxy+y,RED)=1;
data(x,bestmaxy+y,BLUE)=0;
data(x,bestmaxy+y,GREEN)=0;
}
}
for(x=-1;x<=1;x++) {
for(y=bestminy; y<=bestmaxy; y++) {
data(bestminx+x,y,RED)=1;
data(bestminx+x,y,BLUE)=0;
data(bestminx+x,y,GREEN)=0;
data(bestmaxx+x,y,RED)=1;
data(bestmaxx+x,y,BLUE)=0;
data(bestmaxx+x,y,GREEN)=0;
}
}
return temp;
}
// This is the prototype image class.
#include <stdio.h>
class Image {
public:
Image();
Image(int, int, int);
~Image();
void input(FILE*);
void output(FILE*);
void output(FILE*, char*);
void outputPGM(FILE*);
void outputPPM(FILE*);
void outputPGMraw(FILE*);
void outputPPMraw(FILE*);
void outputDAT(FILE*);
void grayscale();
void binarize();
void binarize(double);
void binarize(int);
void equalize();
void crop(int, int, int, int);
void scale(int, int);
void findborder();
Image* edge();
Image* hough(double, int, int);
int& data(int x, int y, int z) {
if(x >= 0 && x < width && y >= 0 && y < width)
return pixeldata[x*height*3 + y*3 + z];
alwayszero = 0;
return alwayszero;
}
// private:
int width, height;
int maxval, alwayszero;
int *pixeldata;
};
#include "image.h"
#include <stdio.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
if(argc < 3) {
printf("Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
printf("Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
printf("Bad output file!\n");
return -3;
}
monkey.input(input);
fclose(input);
monkey.grayscale();
// monkey.equalize();
// monkey.binarize();
// monkey.outputPGM(output);
// monkey.output(output);
monkey.equalize();
monkey.binarize();
ptr = monkey.edge();
// ptr = new Image(240, 320, 255);
// monkey.equalize();
// ptr = monkey.edge();
// ptr -> binarize();
// ptr -> equalize();
// ptr -> hough();
// printf("Um, the thing is %d x %d.\n", ptr->height, ptr->width);
// fflush(stdout);
// monkey.outputPGM(output);
ptr->output(output);
// monkey.output(output);
fflush(output);
fclose(output);
}
// This is a simple linked list implementation
// for lines and pairs of lines (for use in the
// hough transform in the image class)
#include <stdio.h>
#include <malloc.h>
#include "llist.h"
LList *head, *tail;
DLList *headd, *taild;
LList*
appendline(float x, float y, int xi, int xf, int yi, int yf) {
if(head == NULL) {
head = tail = (LList*)malloc(sizeof(LList));
tail->prev = NULL;
}
else {
tail->next = (LList*)malloc(sizeof(LList));
tail->next->prev = tail;
tail = tail->next;
}
tail->r = x; tail->theta = y;
tail->x1 = xi; tail->x2 = xf;
tail->y1 = yi; tail->y2 = yf;
tail->next = NULL;
return tail;
}
void
deleteline(LList *now) {
if(now == head) {
if(now->next == NULL) {
head = tail = NULL;
}
else {
head = now->next;
head->prev = NULL;
}
}
else if(now == tail) {
tail = now->prev;
tail->next = NULL;
}
else {
now->prev->next = now->next;
now->next->prev = now->prev;
}
free(now);
}
DLList*
appenddlines(float x1, float y1, float x2, float y2, int x0, int y0) {
if(headd == NULL) {
headd = taild = (DLList*)malloc(sizeof(DLList));
taild->prev = NULL;
}
else {
taild->next = (DLList*)malloc(sizeof(DLList));
taild->next->prev = taild;
taild = taild->next;
}
taild->r1 = x1; taild->theta1 = y1;
taild->r2 = x2; taild->theta2 = y2;
taild->xi = x0; taild->yi = y0;
taild->next = NULL;
return taild;
}
void
deletedlines(DLList *now) {
if(now == headd) {
if(now->next == NULL) {
headd = taild = NULL;
}
else {
headd = now->next;
headd->prev = NULL;
}
}
else if(now == taild) {
taild = now->prev;
taild->next = NULL;
}
else {
now->prev->next = now->next;
now->next->prev = now->prev;
}
free(now);
}
// simple linked list implementations
// for lines and pairs of lines
typedef struct A {
float r, theta;
int x1, x2, y1, y2;
struct A *prev, *next;
} LList;
typedef struct B {
float r1, theta1;
float r2, theta2;
int xi, yi; // point of intersection
struct B *prev, *next;
} DLList;
extern LList *head, *tail;
extern DLList *headd, *taild;
LList* appendline(float r, float theta, int x1, int x2, int y1, int y2);
void deleteline(LList*);
DLList* appenddlines(float r1, float theta1, float r2, float theta2, int xi, int yi);
void deletedlines(DLList*);
// Binarize an image
#include "image.h"
#include <stdio.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
if(argc < 3) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file!\n");
return -3;
}
monkey.input(input);
fclose(input);
monkey.binarize();
monkey.output(output,argv[2]);
fflush(output);
fclose(output);
}
// Crop an image
#include "image.h"
#include <stdio.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
int minx, maxx, miny, maxy;
if(argc < 3) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file!\n");
return -3;
}
scanf("%d %d %d %d", &minx, &maxx, &miny, &maxy);
monkey.input(input);
fclose(input);
monkey.crop(minx, maxx, miny, maxy);
monkey.output(output,argv[2]);
fflush(output);
fclose(output);
}
// Edgify an image
#include "image.h"
#include <stdio.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
if(argc < 3) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file!\n");
return -3;
}
monkey.input(input);
fclose(input);
ptr = monkey.edge();
ptr->output(output,argv[2]);
fflush(output);
fclose(output);
}
// Equalize an image
#include "image.h"
#include <stdio.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
if(argc < 3) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file!\n");
return -3;
}
monkey.input(input);
fclose(input);
monkey.equalize();
monkey.output(output,argv[2]);
fflush(output);
fclose(output);
}
// Find the border of an image
#include "image.h"
#include <stdio.h>
#include <stdlib.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
if(argc < 3) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file!\n");
return -3;
}
monkey.input(input);
fclose(input);
monkey.findborder();
monkey.output(output,argv[2]);
fflush(output);
fclose(output);
}
// Grayscalify an image
#include "image.h"
#include <stdio.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
if(argc < 3) {
printf("Come on:\n\t%s <input file> <output file>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
printf("Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
printf("Bad output file!\n");
return -3;
}
monkey.input(input);
fclose(input);
monkey.grayscale();
monkey.output(output,argv[2]);
fflush(output);
fclose(output);
}
// Hough transform an image
#include "image.h"
#include <stdio.h>
#include <stdlib.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
* -4 = bad second output file :)
*/
int
main(int argc, char **argv) {
FILE* input, *output, *output2;
Image monkey, *ptr;
double threshold;
int bs, mincounts;
if(argc < 3) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file> [newpic output file] [threshold] [bs] [mincounts]\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file %s!\n", argv[2]);
return -3;
}
if(argc >= 4) {
output2 = fopen(argv[3], "w");
if(output2 == NULL) {
fprintf(stderr, "Bad output file %s!\n", argv[3]);
return -4;
}
}
else {
output2 = NULL;
}
if(argc >= 5)
threshold = atof(argv[4]);
else
threshold = 0.25;
if(argc >= 6)
bs = atoi(argv[5]);
else
bs = 5;
if(argc >= 7)
mincounts = atoi(argv[6]);
else
mincounts = 5;
monkey.input(input);
fclose(input);
ptr = monkey.hough(threshold, bs, mincounts);
ptr->output(output, argv[2]);
// ptr->outputDAT(output);
fflush(output);
fclose(output);
if(output2 != NULL)
monkey.output(output2, argv[3]);
}
// Scale an image
#include "image.h"
#include <stdio.h>
#include <stdlib.h>
/* Error codes: -1 = bad parameters (Dude!)
* -2 = bad input file
* -3 = bad output file
*/
int
main(int argc, char **argv) {
FILE* input, *output;
Image monkey, *ptr;
int newwidth, newheight;
if(argc < 5) {
fprintf(stderr, "Come on:\n\t%s <input file> <output file> <newwidth> <newheight>\n", argv[0]);
return -1; // Dude!
}
input = fopen(argv[1], "r");
if(input == NULL) {
fprintf(stderr, "Bad input file!\n");
return -2;
}
output = fopen(argv[2], "w");
if(output == NULL) {
fprintf(stderr, "Bad output file!\n");
return -3;
}
newwidth = atoi(argv[3]);
newheight = atoi(argv[4]);
monkey.input(input);
fclose(input);
monkey.scale(newwidth, newheight);
monkey.output(output,argv[2]);
fflush(output);
fclose(output);
}
The team logged a total effort of 10880 minutes, which equates to 181:20 hrs.
Dave logged a total of 3180 minutes, Kevin logged a total of 2470 minutes, Ali logged a total of 3220 minutes, and Nick logged a total of 2010 minutes.
As a team, we produced approximately 2860 lines of code. Dave wrote 1100, Kevin wrote 108, Ali wrote 1400, and Nick wrote 250.
The robot needs to be connected to a machine running the camdriver and ariad. For speech the machine needs festival. Here is an example of starting the robot.
festival -server & # start the speech server (ariad through it) cd camdriver ./vidcat & # start the camera driver and listen for clients wanting an image cd ../ariad ./ariad & # start the daemon that controls the robot and accepts movement/say commands
There are no options to vidcat. To use vidcat connect to port 5402 and it will return an image back to you. i.e.
telnet localhost 5402 > monkey.ppm
The ariad accepts commands over port 5401. (540+541, get it?)
The FindNumber utility takes an image and prints to the command line a likely number match for that image and the ratio it found. Feeding the program one of the ideal number images produces that number and a ratio of 1.0. For example,
$FindNumber zero.ppm
$0 1.00000000
Other numbers can be checked in the same way.
$FindNumber a_digit.ppm
Imagesegregate <filename> filename û an image in pbm format
ImageSegregate is a utility that takes an image, presumably, of a number, and attempts to separate the digits. After processing, the minimum and maximum x and y coordinates of each region occupied by a digit is written to stdout for further processing by a cropping utility.
Possible Error Messages: None
robotgui.phtml is a php script and web page designed to operate in conjuction with ariaD for controlling a Saphira robot remotely. You connect to ariaD by specifying the hostname or ip address and port and clicking on the Connect button. If the connection is made, the Connection Status located at the top of the page will read 1. Otherwise you will receive an error. After successfully connecting, you can begin controlling the robot by using the Forward, Reverse, Stop, and rotational buttons. The rotation buttons are labeled û180 to 180 in 15 degree increments. Negative rotation is counter-clockwise and position rotation is clockwise. To capture an image from the webcam, press the Refresh button. You can then view the raw image on the left, a semi-processed image in the middle, and final processed image on the right.
Possible Error Messages: Unable to connect socket
Overall, the project plan proved very useful through out the semester. The beginning of the semester followed the project plan closely. Immediately following spring break, the team began to fall behind and therefor deviate from the project plan. This can be seen by looking at the grey areas in the project plan, which are the orginal dates and times planned for the tasks, or otherwise known as the baseline. The parts in blue are the actual dates and times for the subtasks performed.
