List of addDataSource() Examples

Examples of addDataSource()

gov.nysenate.openleg.model.Agenda.addDataSource()
gov.nysenate.openleg.model.Bill.addDataSource()
gov.nysenate.openleg.model.Calendar.addDataSource()
gov.nysenate.openleg.model.Transcript.addDataSource()
nexj.core.meta.xml.XMLMetadata.addDataSource()
Adds a new data source to the metadata. @param dataSource The data source to add. @throws MetadataException if a data sourcewith the same name already exists.
no.priv.garshol.duke.ConfigurationImpl.addDataSource()
Adds a data source to the configuration. If in deduplication mode groupno == 0, otherwise it gives the number of the group to which the data source belongs.
org.apache.tez.dag.api.Vertex.addDataSource()
Specifies an external data source for a Vertex. This is meant to be used when a Vertex reads Input directly from an external source
For vertices which read data generated by another vertex - use the {@link DAG addEdge} method.If a vertex needs to use data generated by another vertex in the DAG and also from an external source, a combination of this API and the DAG.addEdge API can be used.
Note: If more than one RootInput exists on a vertex, which generates events which need to be routed, or generates information to set parallelism, a custom vertex manager should be setup to handle this. Not using a custom vertex manager for such a scenario will lead to a runtime failure. @param inputName the name of the input. This will be used when accessing the input in the {@link LogicalIOProcessor} @param dataSourceDescriptor the @{link DataSourceDescriptor} for this input. @return this Vertex
org.gudy.azureus2.ui.swt.views.table.TableViewSWT.addDataSource()
org.jboss.portal.identity.metadata.config.DatasourcesMetaData.addDatasource()
org.jrobin.core.RrdDef.addDatasource()
Adds single datasource definition represented with object of class DsDef. @param dsDef Datasource definition. @throws RrdException Thrown if new datasource definition uses already used datasource name.
org.jrobin.data.DataProcessor.addDatasource()

Adds a custom, {@link org.jrobin.data.Plottable plottable} datasource (PDEF).The datapoints should be made available by a class extending {@link org.jrobin.data.Plottable Plottable} class.
@param name source name. @param plottable class that extends Plottable class and is suited for graphing.
org.mortbay.jetty.plus.DefaultDataSourceService.addDataSource()
org.rrd4j.core.RrdDef.addDatasource()
se.sics.mspsim.extutil.jfreechart.DataChart.addDataSource()

Examples of org.apache.tez.dag.api.Vertex.addDataSource()

    }


    Vertex stage1Vertex = Vertex.create("map", ProcessorDescriptor.create(
            MapProcessor.class.getName()).setUserPayload(stage1Payload),
        dsd.getNumberOfShards(), Resource.newInstance(256, 1));
    stage1Vertex.addDataSource("MRInput", dsd);
    Vertex stage2Vertex = Vertex.create("ireduce", ProcessorDescriptor.create(
            ReduceProcessor.class.getName()).setUserPayload(stage2Payload),
        1, Resource.newInstance(256, 1));
    Vertex stage3Vertex = Vertex.create("reduce", ProcessorDescriptor.create(
            ReduceProcessor.class.getName()).setUserPayload(stage3Payload),

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()

    DataSinkDescriptor dataSink = MROutput.createConfigBuilder(new Configuration(tezConf),
        TextOutputFormat.class, outputPath).build();


    Vertex tokenizerVertex = Vertex.create(TOKENIZER, ProcessorDescriptor.create(
        TokenProcessor.class.getName()));
    tokenizerVertex.addDataSource(INPUT, dataSource);


    // Use Text key and IntWritable value to bring counts for each word in the same partition
    OrderedPartitionedKVEdgeConfig summationEdgeConf = OrderedPartitionedKVEdgeConfig
        .newBuilder(Text.class.getName(), IntWritable.class.getName(),
            HashPartitioner.class.getName()).build();

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()


    Vertex mapVertex = Vertex.create("initialmap", ProcessorDescriptor.create(
        MapProcessor.class.getName()).setUserPayload(
        TezUtils.createUserPayloadFromConf(mapStageConf))
        .setHistoryText(mapStageHistoryText)).addTaskLocalFiles(commonLocalResources);
    mapVertex.addDataSource("MRInput", dsd);
    vertices.add(mapVertex);


    ByteArrayOutputStream iROutputStream = new ByteArrayOutputStream(4096);
    iReduceStageConf.writeXml(iROutputStream);
    String iReduceStageHistoryText = new String(iROutputStream.toByteArray(), "UTF-8");

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()

    int numTasks = generateSplitsInAM ? -1 : numMapper;


    Vertex mapVertex = Vertex.create("map", ProcessorDescriptor.create(
        MapProcessor.class.getName()).setUserPayload(mapUserPayload), numTasks)
        .addTaskLocalFiles(commonLocalResources);
    mapVertex.addDataSource("MRInput", dataSource);
    vertices.add(mapVertex);


    if (iReduceStagesCount > 0
        && numIReducer > 0) {
      for (int i = 0; i < iReduceStagesCount; ++i) {

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()

      dsd = MRInputHelpers.configureMRInputWithLegacySplitGeneration(stage1Conf, stagingDir, true);
    } else {
      dsd = MRInputLegacy.createConfigBuilder(stage1Conf, TextInputFormat.class, inputPath)
          .groupSplits(false).build();
    }
    stage1Vertex.addDataSource("MRInput", dsd);


    // Setup stage2 Vertex
    Vertex stage2Vertex = Vertex.create("stage2", ProcessorDescriptor.create(
        FilterByWordOutputProcessor.class.getName()).setUserPayload(TezUtils
        .createUserPayloadFromConf(stage2Conf)), dsd.getNumberOfShards());

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()

      dsd = MRInputHelpers.configureMRInputWithLegacySplitGeneration(stage1Conf, stagingDir, true);
    } else {
      dsd = MRInputLegacy.createConfigBuilder(stage1Conf, TextInputFormat.class, inputPath)
          .groupSplits(false).build();
    }
    stage1Vertex.addDataSource("MRInput", dsd);


    // Setup stage2 Vertex
    Vertex stage2Vertex = Vertex.create("stage2", ProcessorDescriptor.create(
        FilterByWordOutputProcessor.class.getName()).setUserPayload(
        TezUtils.createUserPayloadFromConf(stage2Conf)), 1);

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()


        mapWork = (MapWork) (mapWorkList.get(i));
        conf.set(TEZ_MERGE_CURRENT_MERGE_FILE_PREFIX, mapWork.getName());
        conf.set(Utilities.INPUT_NAME, mapWork.getName());
        LOG.info("Going through each work and adding MultiMRInput");
        mergeVx.addDataSource(mapWork.getName(),
            MultiMRInput.createConfigBuilder(conf, HiveInputFormat.class).build());
      }


      VertexManagerPluginDescriptor desc =
        VertexManagerPluginDescriptor.create(CustomPartitionVertex.class.getName());

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()


    assert mapWork.getAliasToWork().keySet().size() == 1;


    // Add the actual source input
    String alias = mapWork.getAliasToWork().keySet().iterator().next();
    map.addDataSource(alias, dataSource);


    Map<String, LocalResource> localResources = new HashMap<String, LocalResource>();
    localResources.put(getBaseName(appJarLr), appJarLr);
    for (LocalResource lr: additionalLr) {
      localResources.put(getBaseName(lr), lr);

View Full Code Here

Examples of org.apache.tez.dag.api.Vertex.addDataSource()

      if( flowElement instanceof PartitionTap ) // todo: generify
        configBuilder.groupSplits( false );


      DataSourceDescriptor dataSourceDescriptor = configBuilder.build();


      vertex.addDataSource( FlowElements.id( flowElement ), dataSourceDescriptor );
      }


    for( FlowElement flowElement : sinkConfigs.keySet() )
      {
      if( !( flowElement instanceof Tap ) )

View Full Code Here

Examples of org.gudy.azureus2.ui.swt.views.table.TableViewSWT.addDataSource()

          DownloadManager dm = (DownloadManager) iter.next();


          if (!torrentsView.isOurDownloadManager(dm)) {
            tv.removeDataSource(dm);
          } else {
            tv.addDataSource(dm);
          }
        }
      }
    }

View Full Code Here

0 1 2 3 4

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.